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Abstract 

The paper introduces ensembles of accumulate-repeat-accumulate (ARA) codes which asymptotically 
achieve capacity on the binary erasure channel (BEC) with bounded complexity , per information bit, of 
encoding and decoding. It also introduces symmetry properties which play a central role in the construction of 
capacity-achieving ensembles for the BEC with bounded complexity. The results here improve on the tradeoff 
between performance and complexity provided by previous constructions of capacity-achieving ensembles of 
codes defined on graphs. The superiority of ARA codes with moderate to large block length is exemplified by 
computer simulations which compare their performance with those of previously reported capacity-achieving 
ensembles of LDPC and IRA codes. The ARA codes also have the advantage of being systematic. 
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1 Introduction 


Error-correcting codes which employ iterative decoding algorithms are now considered state of the 
art in the field of low-complexity coding techniques. By now, there is a large collection of families 
of iteratively decoded codes including low-density parity-check (LDPC), turbo, repeat-accumulate 
and product codes; all of them demonstrate a rather small gap (in rate) to capacity with feasible 
complexity. 

The study of capacity-achieving (c.a.) sequences of LDPC codes for the binary erasure channel 
(BEC) was initiated by Luby et al. [Tj and Shokrollahi ;2|. They show that it is possible to closely 
approach the capacity of an erasure channel with a simple iterative procedure whose complexity 
is linear in the block length of the code mm. Following these works, Oswald and Shokrollahi 
presented in [3] a systematic study of c.a. sequences of LDPC codes for the BEC. Jin et al. 
introduced irregular repeat-accumulate (IRA) codes and presented a c.a. sequence of systematic 
IRA (SIRA) codes for the BEC jlj. A sequence of c.a. SIRA codes for the BEC with lower encoding 
and decoding complexities was introduced in 0 Theorem 2], All of the aforementioned codes have 
one drawback in common: their decoding complexity scales like the log of the inverse of the gap (in 
rate) to capacity [21 El El El IJ; hence, under iterative message-passing decoding, these codes have 
unbounded complexity (per information bit) as the gap to capacity vanishes. 

In 0, the authors presented for the first time two sequences of ensembles of non-systematic IRA 
(NSIRA) codes which asymptotically (i.e., as their block length tends to infinity) achieve capacity 
on the BEC with bounded complexity per information bit. This new result is achieved by puncturing 
bits and thereby introducing state nodes in the Tanner graph representing the codes. We note that 
for fixed complexity, these codes will eventually (for large enough block length) outperform any 
code proposed so far. However, the speed of convergence happens to be quite slow and, for small 
to moderate block lengths, the codes introduced in fS] are not record breaking. 

In this paper, we are interested in the construction and analysis of c.a. codes for the BEC with 
bounded complexity that also perform well at moderate block lengths. We also would also like 
these codes to be systematic and to have reasonably low error floors. To this end, we make use of 
a new channel coding scheme, called “Accumulate-Repeat-Accumulate” (ARA) codes, which was 
recently introduced by Abbasfar et al. j||. These codes are systematic and have both outstanding 
performance, as exemplified in unnum, and a simple linear-time encoding. After presenting an 
appropriate ensemble of irregular ARA codes, we construct a number of c.a. degree distributions. 
Simulations show that some of these ensembles perform quite well on the BEC at moderate block 
lengths. We therefore expect that irregular ARA codes, optimized for general channels, also perform 
well at moderate block lengths (as is partially supported by some simulation results in [5]). This 
issue is regarded as a topic for further research, while this paper is focused on the BEC. Throughout 
the paper, we consider the encoding and decoding complexity per information bit. 

Along the way, we study symmetry properties of c.a. sequences for the BEC and discover a 
new code structure which we call “Accumulate-LDPC” (ALDPC) codes. We show that c.a. degree 
distributions for this structure can be easily constructed based on the results of (3 Theorems 1, 2]. 
This fact and structure was proposed independently by Hsu and Anastasopoulos m- 

The paper is organized as follows: Section El introduces ARA codes, describes their encoding 
and decoding, and their density evolution analysis for the BEC. Section El introduces symmetry 
properties that play a central role in the construction of c.a. sequences of ensembles for the BEC. 
Section El serves as a preparatory step towards the construction of explicit c.a. sequences of ARA 
codes for the BEC, where their complexity of encoding and decoding stays bounded as the gap 
to capacity vanishes. Section El presents explicit constructions of c.a. sequences of bit-regular and 
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check-regular ARA codes with bounded complexity. Section |H1 focuses on the construction of c.a. 
ensembles of ARA, NSIRA and ALDPC codes (with bounded complexity) based on the ensembles 
of self-matched LDPC codes introduced in the section. Computer simulations for the BEC are 
presented in Section 0 and the superiority of self-matched ARA codes with moderate to large 
block length is exemplified by comparing their performance with those of previously reported c.a. 
ensembles of LDPC and IRA codes from HIHj. Finally, Section |B1 concludes our discussion. 


2 Accumulate-Repeat-Accumulate Codes 


In this section, we present our ensemble of ARA codes. Density evolution (DE) analysis of this 
ensemble is presented in the second part of this section using two different approaches which lead 
to the same “DE fixed point equation”; this equation characterizes the fixed points of the iterative 
message-passing decoder. The connection between these two approaches is used later in this paper 
to state some symmetry properties which serve as an analytical tool for designing various c.a. 
ensembles for the BEC (e.g., ARA, IRA and ALDPC codes). 


2.1 Description of ARA Codes 


k 



Figure 1: Block diagram for the systematic ARA ensemble (’Irr.’ and ’SPC’ stand for ’irregular’ 
and ’single-parity check’, respectively, and II stands for a bit interleaver.) 

ARA codes can be viewed either as interleaved serially concatenated codes (i.e., turbo-like 
codes) or as sparse-graph codes (i.e., LDPC-like codes). From an encoding point of view, it is more 
natural to view them as interleaved serially concatenated codes (see Fig. 0) where the encoding 
process is described in Section 12721 

Since the decoding algorithm of ARA codes is simply belief propagation on the appropriate 
Tanner graph (see Fig. n, this leads one to view them as sparse-graph codes from a decoding 
point of view. Treating these codes as sparse-graph codes also allows one to build large codes by 
“twisting” together many copies of a single small protograph H3HH. In general, this approach 
leads to very good codes with computationally efficient decoders. 

In this work, we consider the ensemble of irregular ARA codes which is the natural generalization 
of the IRA codes from jl]. The ensemble of irregular ARA codes differs slightly from those proposed 
in 0 GDI HQ- For this ensemble, we find that DE for the BEC can be computed in closed form and 
that algebraic methods can be used to construct c.a. sequences. 


2.2 Encoding of ARA Codes 

We describe here briefly the encoding process of the ARA codes in Fig. |T| The encoding of ARA 
codes is done as follows: first, the information bits are accumulated (i.e., differentially encoded), 


3 






















DE 

P 

I 




x i 




x 


2 





Figure 2: Tanner graph for the ARA ensemble. 


and then the bits are repeated a varying number of times (by an irregular repetition code) and 
interleaved. The interleaved bits are partitioned into disjoint sets (whose size is not fixed in general), 
and the parity of each set of bits is computed. Finally, the bits are accumulated for the second 
time. A codeword of systematic ARA codes is composed of the information bits and the parity bits 
at the output of the second accumulator. 

Some slight modifications are used later for our simulations and these details are explained in 
Section 0 In this section, all references to the decoding graph should be taken to imply Fig. [3 and 
all sums are assumed to be modulo-2. 


We will refer to the three layers of bit nodes in the decoding graph as systematic bits, punctured 
bits, and parity bits (the parity bits are named as “code bits” in Fig. [3- Referring to the Tanner 
graph of ARA codes, we designate the systematic bits from left to right by (ui,v, 2 , ■ ■ ■ ,Uk). The 
same convention is used for the punctured bits (v\, V 2 , ■ ■ ■, Vk) and the parity bits (zi,Z 2 ,, z n _k). 


From the upper part of the graph, it follows that Vj = Uj © Vj-i for j €. {2,... , k} and v\ = u\. 
This yields that 

3 

v 3=^2 u i j = 1,2,..., A:. (1) 

2=1 


Let d(i) be the degree of the z-th “parity-check 2” node where the degree is w.r.t. the edges 
connecting the “punctured bit” nodes and the “parity-check 2” nodes, and c(i,j) be the index of 
the punctured bit attached to the j-th edge of the z-th “parity-check 2” node. All the connections 
between the “punctured bit” nodes and “parity-check 2” nodes are described by these two sequences. 
Let the sequence (w\,W 2 , • • •, w n _k) be defined by 


d(i) 

Wj = v c (ij) i = 1,2,... ,n- k. 
j =i 

This can be thought as the sum of the punctured bits which are connected to the i-th “parity- 
check 2” node. From the lower part of the graph we have Zj = Zj -1 © Wj (where wq = 0), and this 
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Zj = y]m j = 1,2,... ,n - k. (2) 

i=l 

From Fig. [21 and equations 0 and 0, one can see that an ARA code is the serial concatenation 
of four simple codes. The first is an accumulate code (upper part of the graph), the second is an 
interleaved irregular repetition code, the third is an irregular single parity-check (SPC) code (which 
is an irregular code due to the varying degrees of the “parity-check 2” nodes), and finally the fourth 
is a second accumulate code (lower part of the graph). 

2.3 Density Evolution of Systematic ARA Ensembles 

We consider here the asymptotic analysis of ensembles of ARA codes where it is assumed that the 
codes are transmitted over a BEC and decoded with an iterative message-passing decoder. Based 
on the density evolution (DE) equations, derived in terms of the degree distributions of these 
ensembles, we consider the fixed points of the decoding process. In the following, we present two 
different approaches for the DE analysis of ARA codes for the BEC which, as expected, provide 
equivalent results. While the concept of the first approach is standard, the second one is helpful 
in establishing symmetry properties of c.a. ensembles for the BEC; these symmetries are discussed 
later in Section 01 


2.3.1 Density Evolution via Message Passing 


An irregular ensemble of ARA codes is defined by its degree distribution (d.d.). Nodes in the 
decoding graph will be referred to by the names given in Fig. [21 Let L(x) = Lzx* be a 

power series where L, denotes the fraction of “punctured bit” nodes with degree-z. Similarly, let 
R(x) = Ylffi ^x 1 be a power series where Ri denotes the fraction of “parity-check 2” nodes with 
degree-z. In both cases, the degree refers only to the edges connecting the “punctured bit” nodes 
to the “parity-check 2” nodes. Similarly, let A(x) = X i xil and p(x) = Pi x%1 form 

the d.d. pair from the edge perspective where A j and pi designate the fraction of the edges which 
are connected to “punctured bit” nodes and “parity-check 2” nodes with degree-z, respectively. 
We also assume that the permutation in Fig. Q is chosen uniformly at random from the set of all 
permutations. The pair of degree distributions of an ARA ensemble is given by (A ,p). 


It is easy to show the following connections between the d.d. pairs w.r.t. the nodes and the 
edges in the graph: 


A(x) = 


L'(x) 


L'( 1) ’ 

or equivalently, since L(0) = R( 0) = 0, then 

fo Mt) dt 


L[x ) = 


fo X (t) dt 


p{x) = 


R{x) = 


R'(x) 

m 

f 0 X P(t) dt 

fo 1 P^) dt 


(3) 


(4) 


The design rate R of the ensemble of ARA codes (see Fig. 0 is computed by expressing the block 
length n as the sum of k systematic bits and kL' {1)/R' (1) parity bits which then yields 


R = 


1 


1 + 


L'( 1 ) ' 
R'( i) 


(5) 
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A random code is chosen from the ensemble and a random codeword is transmitted over a BEC 
with erasure probability p. The asymptotic performance of the iterative message-passing decoder 
(as the block length of the code tends to infinity) is analyzed by tracking the average fraction of 
erasure messages which are passed in the graph of Fig. Q during the I th iteration. The technique 
was introduced in and is known as density evolution (DE). The main assumption of density 
evolution is that the messages passed on the edges of the Tanner graph are statistically independent. 
This assumption is justified by the fact that, for randomly chosen codes, the fraction of bits involved 
in finite-length cycles vanishes as the block length tends to infinity. 

A single decoding iteration consists of six smaller steps which are performed on the Tanner 
graph of Fig. [3 Messages are first passed downward from the “systematic bit” nodes through each 
layer to the “code bit” nodes. Then, messages are passed back upwards from the “code bit” nodes 
through each layer to the “systematic bit” nodes. Let l designate the iteration number. Referring to 
Fig. El let XfP and xip designate the probabilities of an erasure message from the “parity-check 1” 
nodes to the “punctured bit” nodes and vice-versa, let xp and xp be the probabilities of an 
erasure message from the “punctured bit” nodes to the “parity-check 2” nodes and vice versa, and 
finally, let xP and xP be the probabilities of an erasure message from the “parity-check 2” nodes 
to “code bit” nodes and vice versa. 

From the Tanner graph of ARA codes in Fig. [2J an outgoing message from a “parity-check 1” 
node to a “punctured bit” node is an erasure if and only if the incoming message through the other 
edge which connects a “punctured bit” node to the same “parity-check 1” node is an erasure, and 
also the message received from the BEC for the systematic bit which is connected to the same 
“parity-check 1” node is also an erasure. Using the statistical independence assumption, this yields 
the recursive equation 

X P = 1 - (! ~P) (! — ^5 _1) ) • 

It is also clear from Fig. Q that an outgoing message from a “punctured bit” node to a “parity- 
check 2” node is an erasure if and only if all the incoming messages passed through the edges which 
connect the other “parity-check 2” nodes to the same “punctured bit” node are erasures, and also 
the incoming messages which are passed through the two edges connecting the “parity-check 1” 
nodes and the considered “punctured bit” node are erasures. The update rule of the iterative 
message-passing decoder on the BEC therefore implies that 

x ? = ( 4°) 2 

From the graph in Fig. 0 we obtain in a similar manner the following DE equations of the iterative 

= 1 - (! ~P) 

= (4°) 2 a ( 4 - 1] ) 

= 1 - R(l - 4°) (1 - Z = 1,2,... 

(0 

= px 2 

= i - (i - 4°) 2 p{ l -xf) 

= x^ L(x ( p) 

lim x^p = Xi i = 0,1,..., 5. 


message-passing decoder: 

T (0 

x 0 

T (0 

X 1 

T (0 

x 2 

T (0 

X 3 

T (0 
-k 4 

jo 


A fixed point is implied by 
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Now, we can solve for the fixed point by substituting X 2 into X 3 , and then substituting the result 
into X 4 which gives the fixed point equation 


X4 = 1 — 


1 — p 


1 — p R( 1 — x\) 
Likewise, putting xq into £5 gives the fixed point equation 

P L{x 4 ) 


p(l ~x\). 


( 6 ) 


x 5 = 


1 - (1 -p) L{x 4 ) 

and plugging this into xq gives 

xq = 1 - (1 - x 5 )(l -p) = 


V 


(7) 


1 - (1 -p)L[xd) 

Finally, Eqs. ©, 111 ) and the equality x\ = Xq\(x 4 ) give the following implicit equation for x\ = x: 

P 2 X (l - ( i-pfla-x) ) pO- ~ ®)) 


l-(l-p) L [l- (i^fzsy) Pd - x)' 

This equation provides the fixed points of the iterative message-passing decoder. 


= x. 


( 8 ) 


2.3.2 Density Evolution via Graph Reduction 

For ensembles of ARA codes whose transmission takes place over a BEC, the DE fixed point 
equation © can be also derived using a graph reduction approach. This approach introduces two 
new operations on the Tanner graph which remove nodes and edges while preserving the information 
in the graph. 



Figure 3: Graph reduction operation applied to parity-check nodes (left) and bit nodes (right). 

We start by noting that any “code bit” node whose value is not erased by the BEC can be 
removed from the graph by absorbing its value into its two “parity-check 2” nodes. On the other 
hand, when the value of a “code bit” node is erased, one can merge the two “parity-check 2 ” nodes 
which are connected to it (by summing the equations) and then remove the “code bit” node from 
the graph. This merging of two “parity-check 2” nodes causes their degrees to be summed and is 
shown on the left in Figure 03 Now, we consider the degree distribution (d.d.) of a single “parity- 
check 2” node in the reduced graph. This can be visualized as working from left to right in the 
graph, and assuming the value of the previous “code bit” node was known. The probability that 
there are k erasures before the next observed “code bit” is given by p fc (l — p). The graph reduction 
associated with this event causes the degrees of k + 1 “parity-check 2” nodes to be summed. The 
generating function for this sum of k + 1 random variables, each chosen independently from the 
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d.d. R(x ), is given by R(x) k+1 . Therefore, the new d.d. of the “parity-check 2” nodes after the 
graph reduction is given by 

R(x) = f> fc (1 -p)R(x) k+1 = {1 - P) * {X } - (9) 

1 - pR{x) 

A similar graph reduction can be also performed on the “systematic bit” nodes in Fig. [3 Since 
degree-1 bit nodes (e.g., the “systematic bit” nodes in Fig. 0 only provide channel information, 
erasures make them worthless. So they can be removed along with their parity-checks (i.e., the 
“parity-check 1” nodes in Fig. 0 without affecting the decoder. On the other hand, whenever the 
value of a “systematic bit” node is observed (assume the value is zero w.o.l.o.g.), it can be removed 
leaving a degree-2 parity-check. Of course, degree-2 parity-checks imply equality and allow the 
connected “punctured bit” nodes to be merged (effectively summing their degrees). This operation 
is shown on the right in Figure 0 The symmetry between graph reduction on the information 
bits and the parity checks will become important later. Now, we consider the d.d. of a single 
“punctured bit” node in the reduced graph. This can be seen as working from left to right in the 
graph, and assuming the value of the previous “systematic bit” node was erased. The probability 
of the event where the values of k “systematic bit” nodes are observed and the value of the next 
“systematic bit” node is erased by the channel is given by (1— p) k p. The graph reduction associated 
with this event causes the degrees of k+1 “punctured bit” nodes (from the d.d. L(x)) to be summed. 
Hence, the new d.d. of the “punctured bit” nodes after graph reduction is given by 

do) 


After the graph reduction, we are left with a standard LDPC code with new edge-perspective 
degree distributions given by 


L'(x) p 2 X(x) 

W)~ (1-(1 -p)L(x)) 2 
R'(x) (1 — p) 2 p(x) 

R'(l) (l -pR(x)) 2 


( 11 ) 

( 12 ) 


After the aforementioned graph reduction, all the “systematic bit” nodes and “code bit” nodes are 
removed. Therefore the residual LDPC code effectively sees a BEC whose erasure probability is 1, 
and the DE fixed point equation is given by 

A (l — p( 1 — x)) = x. (13) 


Based on dm and m , the last equation is equivalent to (0. 


Remark 1 (The notation of tilted degree distributions). The tilted degree distributions 
A and p which are given in m and m, respectively, depend on the erasure probability of the 
BEC (p). For simplicity of notation, we do not write this dependency explicitly in our notation. 
However, in Section 0 when discussing symmetry properties and replacing p by 1 — p, the erasure 
probability is written explicitly in these tilted degree distributions. 


2.4 The Stability Condition for ARA Codes 

Like the NSIRA codes presented in 0, ARA codes have DE fixed points at both x = 0 and x = 1. 
One can see this by evaluating (0 at these points while assuming that each d.d. function / satisfies 








/(0) = 0 and /(1) = 1. To get decoding started, the d.d. is perturbed slightly by adding degree-1 
parity-checks, pilot bits, and/or systematic bits. For successful completion of decoding, we need 
the fixed point at x = 0 to be stable. To minimize the number of extra bits required to get decoding 
started, it is also useful for the fixed point (prior to the perturbation) at x = 1 to be unstable. 
Although x = 1 is not a fixed point after the perturbation, the the instability condition helps 
prevent the decoder from getting stuck near x = 1. 


The stability and instability conditions are computed by taking the derivative of the LHS of 
® at x = 0 and x = 1. For the fixed point at x = 0 to be stable, we need the derivative to be less 
than unity, and this gives 


p 2 a 2 



2pA , (l) \ 
1 ~P ) 


< 1. 


(14) 


Ensembles without degree-2 bits are unconditionally stable at x = 0. 


For the fixed point at x 
and this gives 


= 1 to be unstable, we need the derivative to be greater than unity, 


(1 -P?P2 ^A'(l) + 


2(l-p)L'(l) 


P 


> 1 . 


(15) 


This condition requires the presence of a non-vanishing fraction of degree-2 “parity-check 2” nodes; 
ensembles not having this property are unable to immediately create new degree-1 checks and may 
therefore get stuck shortly after starting. The instability condition guarantees that, on average, 
more new degree-1 checks are being created than lost when x is close to 1. 


3 Symmetry Properties of Capacity-Achieving Codes 


In this section, we discuss the symmetry between the bit and check degree distributions of c.a. 
ensembles for the BEC. First, we describe this relationship for LDPC codes, and then we extend it 
to ARA codes. The extension is based on analyzing the decoding of ARA codes in terms of graph 
reduction and the DE analysis of LDPC codes. 

3.1 Symmetry Properties of Capacity-Achieving LDPC Codes 

The relationship between the bit d.d. and check d.d. of c.a. ensembles of LDPC codes can be 
expressed in a number of ways. Starting with the DE fixed point equation 

p\(l — p(l — x)) = x (16) 

where p designates the erasure probability of the BEC, we see that picking either the d.d. A or p 
determines the other d.d. exactly. In this section, we make this notion precise and use it to expose 
some of the symmetries of c.a. LDPC codes. 

A few definitions are needed to discuss things properly. Following the notation in [2j, let V be 
the set of d.d. functions (i.e., functions / with non-negative power series expansions around zero 
which satisfy /(0) = 0 and /(1) = 1); this set is defined by 

{ OO 

/ : f(x) = J2 fkX k , x G [0,1], f k > 0, /(0) = 0, /(1) = 1 
k =1 

Let T be an operator which transforms invertible functions / : [0,1] —* [0,1] according to the rule 

Tf(x) = 1 - / _1 (1 -x) 
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where / -1 is the inverse function of /. The function T f is well-defined on [0,1] for any function / 
which is strictly monotonic on this interval, and therefore for any function in V. We will say that 
two d.d. functions / and g are matched if Tf = g (since T 2 f = /, the equality Tf = g implies 
that Tg = /). Finally, let A be the set of all functions / G V such that T/ e V, i.e., 

A±{f:feV, Tfev}. 


The connection with LDPC codes is that finding some / € A is typically the first step towards 
proving that (/, Tf) is a c.a. d.d. pair. Truncation and normalization issues which depend on 
the erasure probability of the BEC must also be considered. When p = 1, many of these issues 
disappear, so we denote the set of d.d. pairs which satisfy CHI by 

Cldpc = {{X,p)eVxV\X(l-p(l-x))=x} 

= { (A, p) | XeA, P = Tx}. 

The symmetry property of c.a. LDPC codes (with rate 0) asserts that 


,, „ symmetry . ,, 

(X,p) e Cldpc --- (p, X) e Cldpc- 


(17) 


One can prove this result by transforming m when p = 1. First, we let x = 1 — p X (1 — y), which 
gives 

X(y) = 1 — /0 _1 (1 — 2 /). 

Then we rewrite this expression as 


p(l - A(y)) = 1 -y 


and let y = 1 — z to get 

p{ 1 - A(1 - z)) = z. 

Comparing this with the DE fixed point equation m when p = 1 shows the symmetry between A 
and p. 


3.2 Symmetry Properties of ARA Codes 


The decoding of an ARA code can be broken into two stages. The first stage transforms the 
ARA code into an equivalent LDPC code via graph reduction, and the second stage decodes the 
LDPC code. This allows us to describe the symmetry property of c.a. ARA codes in terms of the 
symmetry property of c.a. LDPC codes. First, we introduce notation which allows us to express 
compactly the effect of graph reduction on an arbitrary d.d. from the edge perspective (see (@J), 
(fTTl) and (U2J)). For / £ V. let us define 


A (! ~P) 2 f( x ) 

_ P f* f(t)dt y 

fo ) 


(18) 


This allows the graph reduction of an ARA code to be interpreted as a mapping Uara from an 
ARA d.d. pair to an LDPC d.d. pair which can be expressed as 


(Kp) 


Gara 
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The inverse of the graph reduction mapping is represented by a dashed arrow because this inverse 
mapping, while always well-defined, does not necessarily preserve the property of having a non¬ 
negative power series expansion around zero. 


Referring to ensembles of ARA codes, the set of d.d. pairs which satisfy the DE fixed point 
equation ® is given by 

Cara(p) = | (A,p) G V x V | Ai_ p (l - p p ( 1 - x )) = x | 
where the equivalence to © follows from ITT]). (tUD and m 


The symmetry between the bit and check degree distributions of c.a. ARA ensembles follows 
from the symmetry relationship in G2D, and the equivalence between a d.d. pair (A, p) for ARA 
codes and the d.d. pair (Ai - P ,p p ) for LDPC codes of zero-rate. 

The complete symmetry relationship for c.a. ARA ensembles over the BEC is therefore given 
in the following diagram: 


, . „ . . ARA symmetry , 

(A ,p) g Cara(p) - -—* 0, A) 6 C A ra(1 ~p) 


Cara 


Ai -p,Pp) e Cldpc 


LDPC symmetry 


<?ARA 

P p , M-p) £ Cldpc 


The inverse of the graph reduction mapping is represented by the dashed arrow because this inverse 
transformation is only valid if it is known ahead of time that the power series expansions of A and 
p are non-negative. It turns out that this symmetry is very useful in order to generate new d.d. 
pairs which satisfy the DE equality in m- An alternative way to show this symmetry explicitly 
is rewriting m 

Ai_ p (l - p v {x)) = x 

and using the symmetry property m for LDPC codes to rewrite it as 

Pp{ 1 - Ai_p(x)) = 1 - x. 

From m and m, the expansion of the last equation gives 


{1-P?P 1- 


p 2 A(z) 


(l-(l-p)L(x)) 


= 1 — X. 


1 — p R\ 1 — 


p 1 \(x) 


(l—(1 -p)L(x)) 


(19) 


Since the swapping L(x) R(x), X(x) <-»• p(x), p 1 — p. and x <-» 1 — x maps this equation 
back to ©, then we can take any d.d. pair (A , p) which satisfies (JSJ) for p = p* and swap A with p 
(and hence, L and R are also swapped) to get a new d.d. pair which satisfies (H9l) for p = 1 — p* 
(equations @ and (ITT7I should be satisfied for all x G [0,1], so switching between x and 1 — x has 
no relevance). 


3.3 Symmetry Properties of NSIRA Codes 

Now, we consider the graph reduction process and symmetry properties of non-systematic irregular 
rep eat-accumulate (NSIRA) codes (for preliminary material on NSIRA codes, the reader is referred 


11 











to Section 2]). In this respect, we introduce a new ensemble of codes which we call “Accumulate- 
LDPC” (ALDPC) codes. These codes are the natural image of NSIRA codes under the symmetry 
transformation. In fact, this ensemble was discovered by applying the symmetry transformation to 
previously known c.a. code ensembles. Their decoding graph can be constructed from the ARA 
decoding graph (see Fig. 0 by removing the bottom accumulate structure. 

Since an NSIRA code has no accumulate structure attached to the “punctured bit” nodes, the 
graph reduction process affects only the d.d. of the “parity-check 2” nodes. Therefore, graph 
reduction acts as a mapping <7nsira from the NSIRA d.d. pair (A, p) to the LDPC d.d. pair (A, p p ). 
This yields that for ensembles of NSIRA codes, the set of d.d. pairs which satisfy the DE fixed 
point equation is given by 

Cnsira(p) = | (X,p) E V x V I A(l — Pp( 1 - x)) = X j. 

An ALDPC code has no accumulate structure attached to the “parity-check 2” nodes, and 
therefore the graph reduction process only affects the d.d. of the “punctured bit” nodes. Hence, 
graph reduction acts as a mapping aldpc from the ALDPC d.d. pair (A, p) to the LDPC d.d. pair 
(Ai_ p , p). For ALDPC ensembles, the set of d.d. pairs which satisfy the DE fixed point equation 
is therefore given by 

Caldpc(p) = { (A,p) e V x V | Ai_ p (l - p( 1 - x)) = x j. 


The symmetry between c.a. ensembles of NSIRA and ALDPC codes over the BEC follows from 
the symmetry relationship in im, the equivalence between a d.d. pair (A, p) for NSIRA codes and 
the d.d. pair (A, p p ) for LDPC codes, and the equivalence between a d.d. pair (A, p) for ALDPC 
codes and the d.d. pair (Ai - p , p) for LDPC codes. 

The symmetry relationship between c.a. NSIRA and ALDPC ensembles over the BEC is there¬ 
fore given in the following diagram: 


(A ,p) £ Cnsira (p) 


A 


Cnsira ! 

I 

I 

(A, p p ) E Cldpc 


symmetry 


LDPC symmetry 


{p, A) E Caldpc(1 — P ) 

f?ALDPC 

(, Pp , A) E Cldpc 


As before, the inverse of each graph reduction mapping is represented by a dashed arrow because 
this inverse transformation is only valid if it is known ahead of time that the power series expansions 
of A and p are non-negative. 


3.4 Connections with Forney’s Transform 

In [I7j, Forney introduces a graph transformation which maps the factor graph of any group code 
to the factor graph of the dual group code. For factor graphs of binary linear codes which only 
have equality and parity constraints (i.e., no trellis constraints), this operation is equivalent to 
swapping equality and parity constraints (e.g., bit nodes and check nodes). Forney’s approach 
represents observations by half-edges, and these remain attached to the original node even though 
the nature of that node has changed. For example, Forney’s transform maps an LDPC code with 
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parity-check matrix H to a low-density generator-matrix (LDGM) code with generator matrix H 
and the half-edges attached to the bit nodes of the LDPC code are attached to the parity-check 
nodes of the LDGM code. 

Using Forney’s transform, we see that the swapping of A and p described by our symmetry 
mappings actually transforms the original code ensemble into the dual code ensemble. Let the 
design rate of the original ensemble be R, then the design rate of the dual ensemble is 1 — R. This 
means that if we want to have any chance of achieving capacity, we must also map the channel 
erasure probability p to 1 — p. Therefore, our symmetry relationships show that ARA, NSIRA, and 
ALDPC ensembles which are c.a. on BEC under iterative decoding also have dual ensembles which 
are c.a. on the BEC under iterative decoding. 1 In light of the Area Theorem and its relationship to 
the dual code m, this result is not entirely surprising. Still, we had not considered the possibility 
that c.a. ensembles might automatically define c.a. dual ensembles. 

Finally, we note that the basic structure of ARA codes is preserved under Forney’s transform. 
In particular, this means that we can construct self-dual ARA codes (i.e., rate |) by choosing the 
square connection matrix between the “punctured bit” nodes and the “parity-check 2” nodes to be 
symmetric. 


4 Capacity-Achieving ARA Ensembles for the BEC 


This section serves as a preparatory step towards the construction of explicit c.a. ARA ensembles 
for the BEC, whose decoding complexities stay bounded as the gap to capacity vanishes. Later 
in Section El we will present explicit constructions of bit-regular and check-regular ARA ensembles 
which are based on a similar approach due to the symmetry properties provided in the previous 
section. Section El introduces another approach for the construction of c.a. ensembles of ARA codes 
with bounded complexity over the BEC. The concepts used for these constructions are based on 
the symmetry properties in the previous section, and the material presented in this section. 


4.1 A Starting Point for Constructing Capacity-Achieving ARA Ensembles 

Using the tilted degree distributions after graph reduction given in Section 12.3.21 we apply the DE 
equation for LDPC codes m to derive c.a. sequences. This property is proved in the following 
lemma. 

Lemma 1. If equality m is satisfied for all x G [0,1], then the design rate of the corresponding 
ensemble of ARA codes is equal to the capacity of the BEC. 

Proof. From the condition in dm it follows that 

1 — A _1 (l — x) = p{x). (20) 

Since A(0) = 0, A(l) = 1 and A is a monotonic differentiable function on the interval [0,1], then 

1 To be precise, we actually need to consider sequences of ensembles which are c.a. and relate them to sequences 
of dual ensembles. This distinction is rather cumbersome and does not cause problems in this case. 
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from m , the substitution x = X (u) and integration by parts give 


[ p{x) dx = [ ( 

Jo Jo 


1 - 

1 - 

1 - 


1 — A 1 (1 — x)) dx 

[ A _1 (x) dx 

Jo 

/ uX'(u) du 

Jo 

1 Z" 1 — 

u X(u) — / A (u)du 


= / A (u)du. 

Jo 

From ©, (HU), and the equalities L(0) = 0 and L( 1) = 1, we get 

p 2 X(x) 

(1 - (1 -p)L(x)) 2 
1 f 1 p 2 L'(x) 
0)/o 


X(x) dx = 


f 


dx 


L'(l) Jo (l — (1 — p)L(x)Y 


dx 


1 


p 2 du 


Jo (l — (1 — p)u) 2 
P 

L’(iy 

Similarly, from ©, 03) and the equalities R( 0) = 0 and -R(l) = 1, we obtain 


p(x) dx = 


1 — p 

m i)' 


By combining m-m, we obtain the equality 

L'( 1) 


p 


-R'(l) 1 — p 


( 21 ) 


( 22 ) 


(23) 


(24) 


and hence, from the design rate of the ensemble of ARA codes is equal to 1 —p (i.e., the 
ensemble achieves the capacity of the BEC). □ 


Now, consider the DE fixed point equation m (or equivalently ©). Using this equation, we 
see that the condition p(0) = 0 (i.e., no degree-1 “parity-check 2” nodes) is necessary and sufficient 
to guarantee that m is always satisfied at x = 1. Likewise, the condition A(0) = 0 (i.e., no degree- 
1 “punctured-bit” nodes) is necessary and sufficient to guarantee that (Hill) is always satisfied at 
x = 0. From Lemma we conclude that if there exists a d.d. pair (A , p) with bounded average 
degree that satisfies ©, then there is a c.a. sequence of ARA ensembles with bounded complexity 
for the BEC. This conclusion is also based on the truncation discussed in the next section. 


4.2 Truncating Degree Distributions 

After finding a d.d. pair which satisfies the DE equation with equality, a suitable truncation can be 
used to exhibit a sequence of ensembles that achieves capacity. Consider, for example, a sequence 
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of d.d. pairs {(A( M ) ; p( M ))} MeN indexed by the maximum degree M. Since the effect of each 
truncation is negligible as M goes to infinity, Lemma [T| shows that the design rate approaches 
capacity in this case. If the truncations are chosen properly, then we can also show that each 
truncated d.d. pair in the sequence has no DE fixed points for x E (0,1]. This implies that, for 
any 6 > 0, there exists a block length no(5, M ) such that the probability of decoding failure is less 
than <5 for all block lengths n > no. 

Since our DE equations depend on both the edge and node degree distributions, the truncation 
must be chosen carefully to simultaneously bound both. For the check d.d., we want modified 
degree distributions R and p such that R(x) > R(x) and p(x) > p(x) for x E [0,1). In particular, 
we replace large degree checks by degree-1 checks and this gives 

/ oo \ M 

IPl+ Pi) 

\ i=M+ 1 / i=2 

Jo* P(t)dt 

Jo 1 p(t)dt 

This truncation was introduced in [S) and proven to satisfy the desired conditions. 

For the bit d.d., we want truncated degree distributions L and A that satisfy L(x) < L{x) and 
X(x) < X(x) for x E (0,1]. In this case, we replace large degree bits by pilot bits (e.g., these bits 
are forced to zero and known at receiver); this gives 


A(x) 

II 


i —1 


M 

L(x) 

= y Ljx\ 
2—1 


We note that this truncation satisfies the desired conditions (as long as L{ > 0 for some i > M) 
because it simply removes positive terms. 


p(x) = 
R{x) = 


4.3 Encoding and Decoding Complexity 

When transmission takes place over a BEC, the encoding/decoding complexity under iterative 
message-passing decoding is defined to be the average number of edges per information bit in the 
Tanner graph of the code (see Fig. El in p.[lj). The motivation for measuring the complexity in this 
way is because the encoder and the iterative decoder can be both designed to use every edge in the 
graph exactly one time (due to the absolute reliability of information provided by the BEC). 

From the Tanner graph of ARA codes in Fig. [21 it can be verified that the encoding complexity 
(xe) and the decoding complexity (\d) are both equal to 

Xe = Xd = 3 + 1/(1) H————- (25) 

where R is the design rate of the ensemble. 

The complexity of NSIRA codes can also be computed from Fig. [21 by ignoring the accumulate 
structure for the systematic bits. This shows that 

Xe = Xd = L'(l) + —. (26) 
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Likewise, the complexity of ALDPC codes can be computed from Fig. El by ignoring the accu¬ 
mulate structure for the parity bits. This shows that 

3 + L'(l) 

XD= R 

In general, the encoding complexity of ALDPC codes does not grow linearly with the block length 
because it requires the encoding of an LDPC code which can be quadratic in the block length. We 
can, however, apply fast encoding methods for c.a. LDPC codes El to c.a. ALDPC codes. These 
methods will result in linear-time encoding algorithms for c.a. ALDPC codes. 

4.4 The Effect of Puncturing 

Puncturing is a well-known technique that allows one to design for one code rate and adaptively 
increase that rate to match channel conditions. Strictly speaking, we note that punctured ARA 
ensembles are no longer systematic because some information bits may not be transmitted as a 
result of the puncturing. This technique, however, can be used to extend the range of p for which 
certain d.d. pairs are c.a. with bounded complexity. 

For example, consider any code construction which is provably c.a. with bounded complexity 
for p > po (e.g., the check-regular ARA ensemble which will be introduced in Section • This 
construction can be made to achieve capacity for 0 < p < 1 simply by puncturing bits at random 
before transmission (i.e., all bits have the same puncturing rate). Let a be the fraction of bits 
transmitted, then the effective erasure rate of the channel is given by p e ff = 1 — a( 1 — p). Picking 
a < 1 — po guarantees that p e g > po and that the ensemble achieves capacity. This operation does 
increase the complexity by a factor of f because the punctured bits must be retained as part of 
the decoding graph. We apply this method in some computer simulations to increase the code rate 
of a particularly good ensemble of rate \ codes. 

Codes with two classes of bits (e.g., ARA codes) may also benefit from asymmetric puncturing 
of the two classes. For example, puncturing all of the systematic bits of an ARA code converts 
that code into a NSIRA code 0- So we find that sending a fraction a. of the systematic bits of an 
ARA code gives a smooth transition between ARA codes and NSIRA codes for a £ [0,1]. 



5 Bit-Regular and Check-Regular Capacity-Achieving Ensembles 
with Bounded Complexity for the BEC 


This section gives explicit constructions of c.a. ARA ensembles for the BEC, which are either 
bit-regular or check-regular. As will be observed, these ensembles possess bounded complexity (per 
information bit) as the gap to capacity vanishes. 

The symmetry property in Section 13 . 21 allows one for example to design an ensemble of high rate 
ARA codes, and get automatically (by switching between the pair of degree distributions) a new 
ensemble of ARA codes which is suited for low rate applications. We will rely on this symmetry 
property in Section 15.21 when we transform a bit-regular ARA ensemble designed for a BEC with 
erasure probability p £ (0,p*] into a check-regular ensemble designed for p £ [1 — p*, 1). We also 
rely on the fact that the method in Section f-5. II for computing the function R given the function L 
can be easily inverted using the symmetry property. This means that given an algorithm to solve 
for R(x) in terms of L(x ) for a certain po, the inverse algorithm which solves L(x) in terms of R(x) 
is exactly the same, except that po is replaced by 1 — po- 
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5.1 Solving for R{x) in terms of L(x) 


Given L(x), we start with the calculation of X(x) = jjm ■ Then X(x) is calculated from (ITU , and 
p(x) = 1 — A _1 (l — x) is calculated from (11.‘Ill . Combining m and m gives 

= a-p) 2 

P[ R'M ( l-pR(x )) 2 

and by integrating both sides of this equation, we get 

(1 — p) 2 R(x) 


Jo P[t)dt R\ 1) i -pR(xY 

Since R{ 1) = 1, substituting x = 1 in the last equality and solving for /?/(1) gives 

1 ~P 


(28) 


R'W = 


pit) dt 


By substituting the last equality in (EBP , we get 


Q(x) = 


(1 -p)R(x) 
1 — pR{x) 


where 


Using the fact that y = - r 


a Ip P(t) dt 


Q{x) = , 

Jim* 


pz 


z = we 


R(x) = 


solve (ESP for R{x) and get 
Q(x) 


Combining ©, CUP and CUP gives 


p{x) = 


1 - p + pQ{x )' 
R'(x) 


(29) 


(30) 


(31) 


R’{ i) 


p{x) 


(l -p + pQ{x))‘ 


(32) 


As long as we have p( 1) = 1, then evaluating (13211 at x = 1 gives p( 1) = 1. Therefore, there is no 
need to truncate the power series of p. As we noted above, a very similar approach can be applied 
to solve for L(x) in terms of R(x)\ due to the symmetry property, one can simply apply the above 
procedure to a parity-check d.d. R{x) with an erasure probability of 1 — p. 


5.2 Bit-Regular and Check-Regular Capacity-Achieving ARA Ensembles 

The symmetry between bit-regular and check-regular c.a. ensembles of ARA codes follows from 
the symmetry properties presented in Section EH so we choose to focus on a bit-regular ARA 
ensemble. Let X(x) = x 2 , so L(x) = x 3 , and from (ITT1) 


A(x) = 


p 2 x 2 


(l — (1 — p)x 3 ) 


2 • 
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Based on dm we get 


p 1 (x) = 1 — A(1 — x) 

^ p 2 ( 1 — X’) 2 

(l - (1 -p)(l -x) 3 ) 2 


(33) 


This is exactly [HI Eq. (39)] with p replaced by 1 — p and the p switched with A. Therefore, we 
obtain from |S, Theorem 2] that the tilted d.d. p has the form 


2(1 — p)( 1 — x) 2 sin A arcsin 


27(l-p)(l-x)l 
4j? 


p(x) = 1 + 


V3 




(l-p)(l-x)? 


(34) 


Following the procedure of Section EU starting from pop. gives (after some calculus) 


Q(x) = 3(1 ~ 1MX) + 1 ~ t 1 ~ P(^)) 3 

P 1-(1-P)(1 ~P( X )) 


(35) 


where the calculations leading to the expression for Q are detailed in Appendix El Substituting Q 
in into m gives the following expression of the d.d. R: 


R(x) 


I(l_ 

p\ 1 -p + pQ{x)J 

( \ 


1 I 3(x-l)p(.r) _p_ 1 l 1 PW) 

V ‘- p i-(i-p)(i-p(x)) : V 


(36) 


where the function p is given in (EH). It was verified numerically that for p < 0.384, the first 300 
coefficients of the power series expansion of the d.d. R are non-negative; in Appendix 1C. 3. 21 we 
prove that if p < 0.26, then R has indeed a power series expansion about x = 0 whose all coefficients 
are non-negative. According to Lemma H (see n. II 3D . it also holds in general that any d.d. pair 
satisfying J5| has a design rate equal to the capacity of the BEC. It therefore appears that the 
d.d. pair above characterizes a c.a. ensemble of bit-regular ARA codes over the BEC; the capacity 
of the BEC is achieved with bounded complexity for rates greater than 0.616. We note that the 
convergence speed of the degree distribution for the parity-check nodes is relatively fast. As an 
example, for p = 0.3, the fraction of check nodes with degree less than 32 is equal to 0.968. 


Using the symmetry between A and p (see Section El, this also implies that for rates less than 
0.384, the ensemble of check-regular ARA codes with R(x) = x 3 achieves capacity over the BEC 
with bounded complexity. Based on the symmetry property for c.a. ensembles of ARA codes, the 
d.d. L for the check-regular ARA ensemble is obtained from the d.d. R for the bit regular ARA 
ensembles when p is replaced by 1 — p. From © and the symmetry property, the d.d. L for the 
check-regular ARA ensemble which corresponds to R(x) = x 3 has the form 


L(x) 


( \ 


i , 3(x-l)A(») Up 1— (l—A(x)) 3 

V p p i— P (i— x( X) yJ 


(37) 
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where for this ensemble 


A(.x) 


2p(l — x ) 2 sin ^ arcsin 


27p(l—a:)2 
4(1— p) 6 


1 + 


Vs(i-p ) 4 


p( 1—x)^ 


(1 -p) 3 


(38) 


Note that the d.d. A in p8|) is obtained by replacing p by 1 — p in the RHS of El, which finally 
gives the d.d. function introduced in 0 Eq. (15)]. 


5.3 Capacity-Achieving ALDPC Ensembles 


Using the symmetry relationship between NSIRA and ALDPC ensembles from Section f3.31 we find 
that we already have from 2J Theorems 1 and 2] two c.a. ensembles of ALDPC codes. These 
ensembles are based on the bit-regular and check-regular NSIRA ensembles of [Hj. This was also 
observed independently by Hsu and Anastasopoulos 1121- 

Using symmetry, the check-regular NSIRA ensemble gives a bit-regular ALDPC ensemble which 
provably achieves capacity with bounded complexity for p £ (0,1). Since d.d. for small p has long 
tails, one can also use random puncturing to increase the effective erasure rate of the channel, and 
therefore simplify code design. Similarly, the bit-regular NSIRA ensemble gives a check-regular 
ALDPC ensemble which provably achieves capacity with bounded complexity for p £ [j|, l). In 
this case, random puncturing can be used to extend the valid range to (0,1). 

The replacement of p by 1 — p in [S, Eq. (15)] which corresponds to the right d.d. of the check- 
regular NSIRA ensemble, gives the following check d.d. for the bit-regular ALDPC ensemble with 
left degree of 3 


2(1 — p){ 1 — x) 2 sin i arcsin 


27(1—p)(l—p? 
4p 3 


p{x) = 1 + 


V3 


p* 


T 


(39) 


Likewise, the replacement of p by 1 — p in (HI Eq. (10)] which corresponds to the left d.d. of 
the bit-regular NSIRA ensemble, and the substitution q = 3 gives the following bit d.d. for the 
check-regular ALDPC ensemble with right degree of 3 


A(x) 


1 — (1 — x)- 


1 - (1 -p) 1 -3x + 2 1 - (1 



(40) 


It is worth noting that the bit-regular ALDPC ensemble has minimum bit degree of 3. Therefore, 
truncating the check d.d. to finite maximum check degree makes the ensemble unconditionally 
stable. It also shows that minimum distance of should grow linearly with the block length because 
the minimum distance of LDPC codes with a minimum bit dgeree of 3 and a fixed maximum check 
degree grows linearly with the block length m • To prove this rigorously, however, one must also 
consider the effect of the accumulate structure on the minimum distance. 
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6 Capacity-Achieving Ensembles with Bounded Complexity for 
the BEC: Constructions Based on LDPC Codes 


In this section, we introduce another way of constructing c.a. ensembles of ARA codes for the 
BEC. Rather then solving for the function R in terms of the function L (as in Section E3J or 
doing the inverse via the symmetry property, we consider here another natural way of searching 
for c.a. degree distributions. We start by choosing a candidate d.d. pair (A, p) which satisfies 
equation m and test to see if it can be used to construct an ensemble of c.a. ARA codes. The 
testing process starts by mapping the tilted pair (A, p) back to (A, p) via (1111) and and then 
testing the non-negativity of the resulting power series expansions of A and p. 

Following the notation in Section EH it enables one to rewrite m as p = T\ (so the tilted 
degree distributions A and p are matched), and gives a compact description of capacity-achieving 
d.d. pairs of LDPC codes. We note that since T 2 f = / for an arbitrary function / which has an 
inverse, then / E A if and only if Tf E A. Based on m , we obtain that we need to choose the 
tilted d.d. so that A E V and also T\ E V, i.e., we need that the d.d. A (or p) both belong to the 
set A. The reader is referred to [3, Lemma 1] which considers basic properties of the set A and the 
transformation T. 


So far, by choosing A E A (or p E A), we only know that both tilted d.d. have non-negative 
power series expansions. This property does not ensure that both of the original (i.e., non-tilted) 
d.d. A and p also have non-negative power series expansions. Calculation of A and p from the tilted 
d.d. A and p is not straightforward since both equations involve the d.d. L and R which are the 
normalized integrals of the unknown A and p. In order to overcome this difficulty in solving the two 
integral equations, we suggest calculating the tilted d.d. pair w.r.t. the nodes of the graph using 


L (x ) 


fo A(t) dt 
fo Mt) dt ’ 


fo P(t) dt 

£ P(t) dt 


(41) 


The original d.d. pair w.r.t. the nodes (i.e., the original d.d. pair before the graph reduction) can 
be calculated from Eqs. © and m- We obtain that 


L(x) 


L (x) ~ 

P+ (1 ~p)L(x) ’ 


R(x) 


R{x) 

1 — p + pR(x) 


(42) 


and then use equation © to find (A, p). The critical issue here is to verify whether the functions 
L and R have non-negative power series expansions. 


6.1 Capacity-Achieving ARA Ensembles from Self-Matched LDPC Codes 


It is easy to verify that the function 


/(*) 


(1 — b)x 
1 — bx ’ 


0 < b < 1 


(43) 


belongs to the set A and also Tf = /; in the case where Tf = /, the function / is said to be 
self-matched. Therefore, based on m, we examine here whether the choice A(x) = p(x) = 
can be transformed into an ensemble of ARA codes whose degree distributions have non-negative 
power series expansions. From m and dUD , we get 


L{x) = R(x) 


bx + ln(l — bx) 
b + ln(l — b ) 


(44) 
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and from (EH) , we obtain that 


L(x) 
R{x) 


bx + ln(l — bx) 

p[b + ln(l — b)] + (1 — p) [bx + ln(l — 6x)] 
bx + ln(l — bx) 

(1 — p) [b + ln(l — b)} + p [bx + ln(l — bx)]' 


(45) 

(46) 


Since we started with the function / in m which is self-matched, the resulting functions L and 
R in this approach are exactly the same, except that p and 1 — p are switched. In Annendix lC.3l 
it is proved that the degree distributions L and R in ESI) and EH, respectively, have non-negative 
power series expansion if and only if 


1 

i - • (b + ln(l - &)) 


< p < 1 


1 

1 - • (b + ln(l - 6)) ' 


(47) 


Fortunately, there exists a region of ( b,p ) where this condition is satisfied. For the specification 
of this region, we use the Lambert W-function W(x) which is defined to be the u>-solution of the 
equation we w = x; this function is real for x > — i. In the following we introduce and prove the 
following theorem: 

Theorem 1 (Ensembles of Self-Matched ARA Codes). The ensemble of self-matched ARA 
codes, defined by the pair of degree distributions (L, R) in (1451) and EH, achieves the capacity of the 
BEC for any erasure probability p £ (0,1). This result is achieved under iterative message-passing 
decoding with bounded complexity. 

The tails of the d.d. (i.e., the partial sums Y^=k and Ri) decay exponentially like 0{b k ) 
where the parameter b is given in terms of the Lambert W-function as 

/ 13+V6I 1+|1 —2p| 

b = W[—e 12 i-li-2pl +1. (48) 


The complexity, per information bit, of encoding and decoding is given by 


XE = XD 


3 — p b 2 p 

1 —p (1— &)[£> +ln(l — 6)] 


(49) 


Proof. Referring to the pair of degree distribution L and R in EH and EH, respectively, we need 
to obtain the necessary and sufficient conditions which ensure that these two function have non¬ 
negative power series expansion about x = 0. For a given value of b in these degree distributions, 
it is proved in Appendix 1C. 3. 31 that this property is satisfied if and only if the inequality in EH 
holds. 


The encoding and decoding complexities of c.a. ensembles of ARA codes for the BEC are 
discussed in Section roi Since our ensemble is c.a., then R = 1 — p where p designates the erasure 
probability of the BEC, and from EH and EH 


L'(l)=pL'(l) = 


b 2 p 

(1 -b)[b + ln(l - b)\ 


Combining EH with the last equality provides the expression in EH for the complexity, per 
information bit, of encoding and decoding. 


For fixed p £ (0,1), the complexity in (1491) forms a monotonic increasing function of b (which 
becomes unbounded as b —► 1 _ ). In order to minimize the encoding/decoding complexity, we wish 
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to find the smallest value of 6 in the interval (0,1) so that the power series expansions about zero 
of the degree distributions L and R are both non-negative. For a fixed value of p, it is equivalent 
to solving for the minimal value of b £ (0,1) which satisfies the condition in (14711 . This gives the 
equation 


1 


= min(p, 1 — p) 


which can be rewritten as 


— b — ln(l — b ) 


13 + ^61 1 + 11 — 2p| 

12 1 — |1 — 2 p\ 


(50) 


by using the equality 


min(p, 1 — p) 


1 

2 



For a 6 M, the solution of the equation —b — ln(l — b) = a is given by b = W (—e _1 ~ a ) + 1. To verify 
this, one needs to write the equation in the form (6 — l)e ft_1 = —e _a_1 , and rely on the definition 
of the Lambert W-function. Hence, the solution of equation m is given by the expression for b 
in (1481) . For p£ (0,1), the expression for b in (14811 achieves its global minimum at p = and its 
value is 


b* 


W(—e~ 


25+v'61 

12 


) + l 


0.9304. 


Eq. (14811 therefore implies that for 0 < p < 1, the parameter b ranges in the interval [b*, 1); it 
achieves the value 6 = 5* at p = and tends to 1 when p approaches zero or unity. 


The asymptotic behavior of the two d.d. pairs w.r.t. the nodes and the edges is derived in 
Appendix IB.31 and is given by 


L k ,R k - O ( fcln2(fc) ) > ^,Pk - O ( ln2(A:) ) 
so the tails of the d.d. pair (L, R ) decay exponentially with k. 
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a-0.5 
0.4 
0.3 
0.2 
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U 0.94 0.95 0.96 0.97 0.98 0.99 1 

b 



(51) 

□ 


Figure 4: Valid values of p as a function of the parameter 6 in (ESI and Enr so that the coefficients 
in the power series expansions around zero of L(-), R(-), A(-) and p{ ) are all non-negative. This 
region is determined by the inequalities in dUD , and it is therefore bounded by the upper and lower 
curves in this figure. 
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The region (b,p) characterized by the inequality in (1471) is depicted in Fig. and we point out 
that its width grows as b gets closer to 1. Note that it follows from (EH that in the limit where 
b —> 1~, the d.d. pair L and R have non-negative power series expansions for 0 < p < 1. However, 
from m, the complexity in the limit where 6 —> 1 becomes unbounded. 

An efficient algorithm for the calculation of the d.d. pair in and El w.r.t. the nodes of 
the graph, and the d.d. pair (A , p) w.r.t. the edges is given in Appendix IB. 21 

We believe the performance advantage of this ensemble over other c.a. ensembles is mainly due 
to the exponential decay of the d.d. coefficients, as given in m- 


Numerical Results 

In Fig. [2J we show the encoding and decoding complexity of the self-matched ARA ensemble 
introduced in this section (left plot), and the minimal value of k such that the partial sums Yli=2 
and Yli= 2 Pi excee d 0.95 (right plot). 




Figure 5: Left plot: Encoding/decoding complexity per information bit of the considered ensemble 
of ARA codes. The complexity is given in El as a function of the erasure probability (p) of the 
BEC, and to this end, we choose the parameter b according to equation El for minimizing the 
complexity. Right plot: A plot of the minimal value of A; as a function of the erasure probability 
(p) so that the partial sums JT _ 2 A * and Yli =2 Pi both exceed 0.95. We choose the parameter b 
according to equation El which also minimizes k for a fixed value of p. 

Both of these sums converge to 1 as A: goes to infinity, and the convergence time is measured by 
the minimal value of k where these partial sums exceed a threshold which is close to 1 (e.g., 0.95). 
We note that, like the complexity, the convergence time of these partial sums is increasing with b. 
Therefore, the choice we made according to equation El minimizes both quantities simultaneously. 
Notice that the complexity and convergence time are rather small for 0.35 < p < 0.65. Both of 
these quantities achieve their minimal value at P = \ since the value of b required by El is also 
minimized. We also note that the minimal value of k is symmetric around p = -7 (see lower plot) 
while the complexity is asymmetric (see upper plot). The reason for the symmetry property around 
p = \ in the lower plot of Fig. 0 is because the replacement of p by 1 — p yields the same value of b 
in El; actually, the replacement of p by 1 — p yields the same d.d. pair, except that {\i}i >2 and 
{Pi}i> 2 are switched (this follows directly from (H51) and El)- 

For p = 0.50 and p = 0.60, the convergence rates of the degree distributions w.r.t. the nodes 
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Figure 6: The plots refer to the ensemble of self-matched ARA codes with the pair of degree 
distributions in m and (p|): these plots show the partial sums Yli= 2-^0 Yli= 2^0 and 

Yhi= 2 Pi as a function of the integer k. The upper plot refers to p = 0.500 and b = 0.9304 (where for 
this case, A* = Ri and A i = pt for all i £ N), and the lower plot refers to p = 0.600 and b = 0.972. 
The upper and lower plots refer to the (encoding and decoding) complexity per information bit 
which, under message-passing iterative decoding, is equal to 8.585 and 13.776, respectively. 

and edges of the graph are shown in Fig. [HI The value of the parameter b is determined by 
(TTS1) . and the corresponding complexity of encoding and decoding is equal to 8.585 and 13.776, 
respectively (the complexity is therefore relatively small in both cases). The results for a BEC with 
erasure probability p = 0.50 (i.e., a design rate of ^) are particularly encouraging. In this case, 
the encoding/decoding complexity is equal to 6.585, and the partial sum Yli =2 ( or equivalently, 
'£2i=2Pi) excee ds 0.95 for k > 29. For comparison, consider the check-regular NSIRA ensemble 
in [HI Theorem 2] which requires more than 300 terms so that the partial sum Yli =2 exceeds 
0.95. This significant improvement in the convergence rate of the degree distributions yields codes 
whose performance for moderate block lengths is superior to previous constructions. The considered 
ensemble of self-matched ARA codes has the property that for a design rate of one-half, L = R 
and A = p, so the d.d. pairs of the punctured bits and the parity-checks coincide. 
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In Fig. Cl we compare the asymptotic expressions of the degree distributions {L^.}, {i?*,}, {A^}, 
{pk} to their exact values. There is a good match between the asymptotic and exact values for 
moderate to large values of k. The best match between the two expressions is obtained when 
p = 0.50 because this affords the minimal value of b. To see this phenomenon exactly, one can look 
at the error terms of the asymptotic expressions given in Appendix IB.31 



Figure 7: A plot comparing the asymptotic versus exact expressions of the degree distributions 
{Lk}, {-Rfc}, {Afc} and {pk] of the self-matched ARA ensemble, designed for a BEC with erasure 
probability of p = ^. The parameter b in the left and right degree distributions (ESI) and (©, 
respectively, is determined by Eq. EH), so its value is b = 0.9304. 

To conclude, we note that the ensemble of self-matched ARA codes without puncturing, as 
considered in this section is well suited for moderate rates while, on the other hand, the ensembles 
of bit-regular and check-regular ARA codes are well suited for high and low rates, respectively. In 
order to make the ensemble of self-matched ARA codes suitable for high code rates, we use random 
puncturing (as will be exemplified later, the performance of these ensembles with puncturing is 
good also for moderate block lengths). 

6.2 Further Discussion on the Construction of ARA Codes From LDPC Codes 

The starting point in Section 16.11 was the choice of the function / in (14311 which belongs to the set 
V and which also satisfies the property / = Tf. This choice simplifies the analysis in Section 16.11 
by setting \{x) = p(x) = f(x). From the symmetry property stated in Sectional we see that L 
and R (and also A and p) have the same form except that p is replaced by 1 — p. The function / 
in (14311 is however not the only function in the set V which satisfies the property f = Tf. In jHl 
Appendix V], there is a discussion on the fixed points of the operator T. We cite here a necessary 
condition for the satisfaction of this property. 

Proposition (0 Appendix V]): If / = T/ for some / £ V, then f(x) = t(x) + 1 — x where t(x ) 
satisfies the identity 

t(x) = 2x — 1 — h(t(x)), x £ [0,1] (52) 

and the function h satisfies the properties 

0 < h(t) < 1, h(t) = t £ [0,1], h{ 1) = h{— 1) = 0. 
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Choosing appropriate functions in (Pll may lead to new families of fixed points of the operator T. 
For example, the choice h(t) = a(l — t 2 ) gives the function 


/(*) 



(53) 


which indeed belongs to the set V and satisfies the property f = Tf for 0 < ct < Using an 
approach similar to the one in Section 16.11 we set A = p = /, and rely on m to calculate L and 
R. These functions are identical because / is a fixed point of the operator T. The choice of the 
function in PI) gives 


L(x) = R(x) 


fo fit) dt 

fo f(t) dt 

(1 + e)*"T + § 


(1 + ^ f ) 2 


1 _ 2 a 

2 3 


2x 

a 




and the candidate d.d. pair ( L,R ) is then calculated by using (1421) . From our numerical ex¬ 
periments, L is likely to have a non-negative power series expansion for p E [0.85,1) (with a E 
[0.38,0.5]). As we noted above, the expressions for L and R are the same except that p is replaced 
by 1 —p. Since the intervals [0.85,1) and (0,0.15] do not overlap, we find that both L(x) and R(x) 
cannot simultaneously have non-negative power series expansions about x = 0. This shows that 
not all self-matched functions give rise to valid degree distributions of ARA ensembles. 


6.3 Capacity-Achieving NSIRA Ensembles from Self-Matched LDPC Codes 


In this section, we construct ensembles of NSIRA codes using LDPC codes whose degree distribu¬ 
tions from the edge perspective are matched. We apply here the concept of DE via graph reduction 
to ensembles of NSIRA codes. In this case, the graph reduction only applies to the “parity-check 2” 
nodes (see Fig. |2J). This is because the upper part of Fig. [21 does not exist in the Tanner graph of 
NSIRA codes (i.e., the “punctured bit” nodes in this figure are the “information bit” nodes in the 
graph of NSIRA codes). Based on graph reduction, we obtain that L = L for ensembles of NSIRA 
codes, while the functions R and R satisfy the equality in (PI). In a similar manner, the equality 
A = A holds for NSIRA ensembles while equality m is satisfied for the degree distributions of the 
parity-checks from the edge perspective. We note that from m and m, the fixed point of the 
DE equations for NSIRA ensembles is given by 


/_ (i-riva-xD 

V (l-pR(l-x)) 2 j 

Of course, this equation coincides with the DE fixed point equation [HI Eq. (6)] (with xq replaced 
by x) derived previously for NSIRA codes. 


For the construction of ensembles of NSIRA codes using LDPC codes whose degree distributions 
from the edge perspective are both matched to themselves, we rely as a starting point on the function 
/ in (l43l) which forms a d.d. which is matched to itself, and set A(x) = p{x) = ^ for 0 < b < 1, 
similarly to Section 16.II For the considered ensemble of NSIRA codes, the d.d. L(x) is then equal 
to L(x) in (TH1) . i.e., 


bx + ln(l — bx ) 
b + ln(l — b) 


(54) 
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From this, we see that there are no degree-1 “information bit” nodes, and that the fraction of 
“information bit” nodes with degree-* is given by 


Li = 


V 1 

i b + ln(l — b) 


i — 2,3,... . 


The non-negativity of the sequence {Li} holds when 0 < b < 1 (so b + ln(l — b) < 0). Therefore, 
the power series expansion of the d.d. L is always non-negative and there is no requirement on the 
erasure probability p in this regard. The condition for the d.d. R to be non-negative is identical to 
that of the self-matched ARA codes derived in Appendix 1C. 3. 31 and is given by 


1 _ 13^61 [ fe + in(! _ 6 )] ' 

By comparing it to the parallel requirement for the ARA ensemble, as given in 6TP, one observes 
that O requires a weaker condition on p which is only the upper bound on p in (|17|). As mentioned 
above, the d.d. R is the same as for the ARA ensemble in Section Kill The encoding and decoding 
complexities of this ensemble are equal and have the form 


XE = XD 


2 b 2 

1 — p (1 — b) [b + ln(l — 6)] 


(56) 


This gives an explicit construction of NSIRA ensembles from LDPC codes whose degree dis¬ 
tributions from the edge perspective are matched to themselves. In general, we find by computer 
simulations for finite-length codes over the BEC that ARA codes have the best performance. 


6.4 Capacity-Achieving ALDPC Ensembles from Self-Matched LDPC Codes 


In this section, we construct ensembles of ALDPC codes using LDPC codes whose degree distri¬ 
butions from the edge perspective are matched. Using the symmetry property between NSIRA 
and ALDPC codes, this construction follows almost trivially from the results of Section 16.31 The 
symmetry transformation acts by switching A with p and p with 1 — p, which gives 


R(x) 


bx + ln(l — bx) 
b + ln(l — b) 


(57) 


From this, we see that there are no degree-1 “parity-check 2” nodes and that the fraction of “parity- 
check 2” nodes with degree-* is given by 


R, = 


b l l 
i b + ln(l — b) 


i — 2,3,... . 


The non-negativity of the sequence {R*} follows directly because 0 < b < 1 (so b + ln(l — b) < 
0). Since the Ri are non-negative for p E [0,1], the the valid range of p for this construction is 
determined by the non-negativity of L(x). Similar to the NSIRA ensemble in Section 16.31 the d.d. 
L is equal to the one in (1451) for the ARA ensemble. From Appendix IC.3.31 we see that L(x) will 
have a non-negative power series expansion if 

p >- -=^- - . (58) 

1 - 13 -/^ [b + ln(l - b )] 

Using (EH and the decoding complexity of this ensemble is given by 


_3_ tfp _ 

1 —p (1 — p){l - b)[b + ln(l - b)} 


(59) 
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This gives an explicit construction of ALDPC ensembles from ensembles of LDPC codes with 
self-dual degree distributions. In Sectional we compare the performance of this ensembles with the 
ensembles of NSIRA and ARA codes in Sections O and 16.31 respectively. In general, we find that 
the ARA codes have the best performance. 


7 Computer Simulations 


In this section, we describe the details of our computer simulations and discuss the results. In 
particular, our main result is that the ARA ensemble of Section El shows a distinct advantage over 
all the other c.a. ensembles we consider. We believe this advantage is largely due to the exponential 
decay of its d.d. tails. 


7.1 Encoding Details 

In this section, we consider the explicit construction of finite-length ARA codes from infinite degree 
distributions. Let k be the number of “systematic bit” nodes and n — k be the number of “parity- 
check 2” nodes (see Fig. El in p.EJ- In the first step, we scale and quantize the degree distributions 
into a list of integers corresponding to the number of “punctured bit” nodes and “parity-check 2” 
nodes of each degree. All “parity-check 2” nodes with degree greater than dn are reduced to degree- 
o!r . This truncation has the effect of reducing the number of edges between “parity-check 2” nodes 
and “punctured bit” nodes. All “punctured bit” nodes with degree greater than di, are converted 
into pilot bits. Therefore, the values of these bits are forced to a known value (e.g., zero) at the 
encoder. This is done by choosing the value of one of the “systematic bit” nodes carefully as 
described in Section fO 

We note that each pilot bit has the effect of reducing the dimension of 

the code by 1. 

The main reason for introducing pilot bits is that decoding cannot get started without them 
(see the DE equations in Section Eh 3. II) . A slight modification of the encoding process is required 
for pilot bits, and we describe it now using the notation from Section 12.21 Our goal is to set the 
punctured bit Vj to zero by picking a systematic bit appropriately. We find that the punctured bit 
Vj can be set to zero by choosing the systematic bit Uj to be Vj-i = J2i=i u i- Since each pilot bit 
determines one of the systematic bits, this process reduces the actual number of information bits 
by the number of pilot bits. 

While long random codes chosen from the ensemble of Section 16.11 tend to have vanishing bit 
erasure probability as the block length goes to infinity, the block erasure probability does not 
vanish. This phenomenon is caused by small weaknesses in the graph, and can be mitigated by 
using a high-rate outer code as a second layer of protection. In the case of ARA codes, we believe 
that it is most effective to apply this code to the punctured bits. In particular, we force the last 
m punctured bits to equal random linear combinations of the first k — m punctured bits. This 
operation reduces the information content of the code by m bits. 

Again, we must slightly modify the encoding process to accommodate the outer code. Using 
the notation of Section [Q we find that this involves forcing the sequence v to satisfy Hv = 0 for 
some m x k parity-check matrix H. For our purposes, it suffices to consider matrices of the form 
H = [PI], where I is the m x m identity and P is an m x {k — rri) matrix with entries Pj : {. We 
can require the v sequence to satisfy Hu = 0 by choosing each Uj, for j e {k — m + 1,..., k}, as 
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follows. We choose 


k—m 

u j = v j -1 + P i’ iVi 

i =1 

because combining this with 0 shows that 

k—m 

Vj = ^2 Hv = 0. 

i= 1 


7.2 Decoding Details 

The simulations essentially use Luby’s decoding algorithm |T|, which starts with the full graph and 
deletes edges as they become known. This process starts by choosing a degree-1 parity-check node 
and finding the bit node to which it attaches. This bit node is declared known, and all of the edges 
which are attached to it are removed from the graph. Without loss of generality, one can assume 
that the all-zero codeword was transmitted, and the received bits are erased with probability p. 

One advantage of Luby’s decoding algorithm for LDPC codes is that it averages over all random 
graphs without explicitly constructing each graph. It does this simply by tracking the number of 
nodes of each degree throughout the decoding process. For ARA codes, we first use graph reduction 
(from Section 12.11.21) to convert the ARA graph into an LDPC graph. This reduction is not done 
in the average sense, but instead by explicitly placing erasures and combining nodes. This step 
implicitly averages over all orderings of the systematic and parity bits. Next, the resulting LDPC 
code is decoded using Luby’s algorithm. This step implicitly averages over all random graphs. 
Therefore, this simulation technique averages over the entire ARA ensemble. Careful design of the 
graph can only improve performance. 

If decoding terminates before all bits are known, then the high-rate outer code is decoded. 
Assume that l bits remain unknown after iterative decoding finishes. Decoding the outer code is 
equivalent to solving a set of m linear equations with l unknowns. Of course, the parity checks 
leftover from iterative decoding can be used to increase the number of equations (or reduce the 
number of unknowns). In particular, it is easy to use one of the leftover degree-2 parity-check nodes 
to reduce the number of unknowns by one. Such a parity check implies the equality of the two 
unknowns connected to it. This equality allows one to add two columns of the matrix together and 
reduce the number of unknowns by one. This process can be continued, but one must be wary of 
linear dependencies among the degree-2 parity-check nodes. Therefore, if t is the number of linearly 
independent degree-2 parity-check nodes remaining at the end of iterative decoding, then decoding 
is successful if and only if the rank of the new mx (l — t) matrix is l — t. We note that each entry in 
the P matrix is chosen randomly from {0,1} with equal probability. Choosing the P matrix more 
carefully can only improve performance. 


7.3 Discussion of Results 

In this section, we discuss the results of the computer simulations. Although all of the ensembles 
introduced have been simulated, results are presented mainly for the ARA and IRA ensembles which 
are constructed from self-matched LDPC codes. These codes seem to have the best performance 
in the waterfall region (probably because their degree distributions decay exponentially fast). The 
results are compared with simulations of Shokrollahi’s check-regular ensemble j2j. 
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Figures [HI and El compare the self-matched ARA, the self-matched IRA, and the check-regular 
LDPC ensembles at rate ^. Figures |H1 shows the raw error rates without using a high rate outer 
code. Therefore, the error floor is rather severe because the results are averaged over the entire 
ensemble and no attempt was made to avoid small stopping sets. Figure El shows the results when 
a high rate outer code is used to mitigate small stopping sets. The results show that the self- 
matched ARA ensemble can handle an erasure rate roughly 0.005 larger than the other ensembles 
while maintaining the same performance. This gain is present throughout the waterfall region and 
similar for block lengths of 8192 and 65536 bits. 

Fisr. llOl shows how the performance of the self-matched ARA ensemble varies with block length. 
The upper plot shows the results without a high rate outer code and one notices that the word 
erasure probability never goes below 10%. Even with the high rate outer code, this ensemble has a 
word erasure floor due to the fraction of degree-2 bit nodes. As with stable ensembles of irregular 
LDPC codes, this floor can be made arbitrarily low by expurgating low weight stopping sets and/or 
adding a stronger outer code. 

For a rate R > one can either design self-matched ARA codes directly for this rate or, 
alternatively, by first designing rate-^ self-matched ARA codes and puncturing the code bits up 
to rate R. The problem with designing the code directly for rate R is that the parameter b in m 
and m must be increased in this case and becomes very close to 1. This increases the encoding 
and decoding complexities and the required maximum degree (see Fig. 0 . For example, the rate-^ 
ensemble requires only about the 30 first terms of the degree distributions in order to achieve 99% 
of the design rate while the rate-^, ensemble requires about the 160 first terms of these degree 
distributions. Figure fTTl shows the performance of these two design methods for rate . The results 
are compared directly in Fig. El showing the advantage of the methodology where the ensemble is 
designed for rate one-half and then punctured to obtain the higher rate. This advantage over the 
approach of designing self-matched ARA codes without puncturing is exemplified in Fig. IIII either 
if the ARA code is combined with a high-rate outer code or not. 

It can be observed from Fig. 0 that the ensemble of self-matched ARA codes with the pair 
of degree distributions in m and m is not suitable for designing codes of low rates; the lower 
the design rate becomes below ^ bits per channel use, the complexity of this ensemble increases 
significantly (see the left plot of Fig. 0 . To this end, we propose the bit-regular accumulate-LDPC 
(ALDPC) codes (see Section RT~11) as a preferable alternative for designing codes of low rates. The 
performance of the bit-regular ALDPC ensemble, where the degree of the bit nodes is set to 3, is 
shown in Fig. El In this figure, the performance of these codes is exemplified for moderate to large 
block lengths, showing the significance of a high-rate outer code in reducing the erasure floor. 
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8 Summary and Conclusions 


In this section, we provide a table of c.a. code constructions for the BEC. With the exception 
of the LDPC and systematic irregular repeat-accumulate (SIRA) codes, all of these codes achieve 
capacity on the BEC with bounded complexity per information bit. Since each of the actual c.a. 
degree distributions has infinite support, we let M be the truncation depth of the d.d. and give 
the gap to capacity as a function of M. 


Code 

Type 

Range of p 

Bit d.d. 

Check d.d. 

Gap to 
capacity (g:) 

Complexity 

as e —> 0 

LDPC 

Tornado 

(0,1) 

A ( x ) = “hM 1 ~ x ) 

p(x) = 


XD = O ^log i j 

LDPC 

CR-fc 

(0,1) 

\(x) = 1 — (1 — x) k ~ 1 

p(x) = x k ~ 1 


Xd = O ^log i j 

SIRA 

0 Th. 2] 

(0,1) 

13 Eq. (33)] 

p(x) = e“( I_ 0 

°(i) 

Xe = Xd = O ^log i j 

NSIRA 

CR-3 

(0,0.95) 

1 Eq. (15)] 

p(x) = X 2 

°(^i) 

5 

XE = XD = ; 

1 -p 

NSIRA 

BR-3 

(o-fk) 

A(x) = x 2 

0 Eq. (10)] 

°{vm) 

2 

XE = XD = 3 + 

1 -p 

NSIRA 

SM 

Eq. 1551 

Eq. 1541 

Eq. 1161 

„ ( h M \ 

° \ln 2 (M)J 

Eq. 1561 

ARA 

CR-3 

(0.616,l)t 

Eq. 1371 

R(x) = x 3 

o(-h) 

Vm5 J 

5 p 

Xe = Xd = 3 + 

1-p 

ARA 

BR-3 

(0, 0.348) 

L(x) = x 3 

Eq. 1361 

o(A) 

2 p 

XE = XD = 6 + 

1-p 

ARA 

SM 

Eq. 1171 

Eq. 1451 

Eq. 1461 

„{ h M \ 
C Aln 2 (A/)J 

Eq. 1491 

ALDPC 

CR-3 

(liT 

Eq. {HJ 

p{x) = X 2 


3(1 +P) 

XD = ! 

1-p 

ALDPC 

BR-3 

(0.05,1) 

A(x) = x 2 

Eq. m 

°(tr) 

6 

XD = , 

1-p 

ALDPC 

SM 

Eq. 15811 

Eq. 1451 

Eq. 1571 

/ b M \ 

Eq. 1591 


Table 1: List of capacity-achieving (c.a.) codes for the BEC. In this table, ’BR’, ’CR’ and ’SM’ 
stand for ’bit-regular’, ’check-regular’ and ’self-matched degree distributions’, respectively. The 
valid range of p is marked by f if it can be extended to (0,1) via puncturing. The parameter b in 
the d.d. pairs of the SM ensembles is allowed to be in the interval (0.9304,1). 

Simulation results show that among all these ensembles, the self-matched ARA ensemble has 
a superior performance for moderate to large block lengths (considering rates which are at least ^ 
bits per channel use). We believe the performance advantage of the self-matched ARA ensemble 
is mainly due to the exponential decay of the d.d. coefficients. For designing low-rate codes, we 
propose the bit-regular accumulate-LDPC (ALDPC) codes in Section lCH as a preferable alternative. 
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Appendix A 
Derivation of Q in (Hbl) 


Following the procedure of Section 15.11 and starting from (IHUll gives 

Jo P(t) dt 


Q(x) = 


Jo 1 P(t) dt 

~i 

P Jo 


p(t ) dt 


where the last equality follows from m and (22), i.e., 

f pit ) dt = f A (t) dt = 

Jo Jo 


L'( 1 ) 


The substitution t = p 1 (ri) and integration by parts gives 


/ P(t) dt 

Jo 

rp{x) r] 

= i "4 (r ( “» du 

rp(x) 

/ p iu)d 

Jo 


= up l {u 


= xp(x) — 


p(x) 

u =0 
r p{x) 


1 0 


1 - 


u 

p 2 (l-u) 2 


= (x- 1 )p[x) - 


(1 - (1 -p)(l - w) 3 ) 

p 2 1 


3(1 — p) 1 — (1 — p)(l — u ) 3 


du 


p{x) 


It—0 


/ s. P 1-(1 - p(x)) 3 

= (x- 1 )p(x) + -- 


3 1 - (1 -p)(l - p(x)) 
and the substitution of the last equality in the RHS of CO gives (1531) . 


(A.l) 
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Appendix B 

Capacity-Achieving Self-Matched Ensembles of ARA Codes 


From (SSJ), we get 

L(x ) = 


1 — p 


1 — p 


1 - 


1 


1 + 


(1 —p) bx-\-\n(l—bx) 


p |&+ln(l— b )j ) 


1 - 


1 


1 — a(p , b) bx + ln(l — bx) 


- -^ ~2a m (p,b ) bx + ln(l — bx) 

P m= 1 


1 °° 

T—pY. {(-i) m - 1( x m (p, b ) 


m =1 


b 2 x 2 b 3 x 3 6 4 x 4 

—- + + ••• 


where 


a(p,b) = -- 


1 — p 


(B.l) 


(B.2) 


p [b + ln(l — b)} 

We note that a(p,b ) is positive for 0 < b < 1 and 0 < p < 1. From equation m, it is easily 
verified that the minimal degree in the power series expansion of bx + ln(l — bx) is x 2m , so if 
k < 2m, then the coefficient of x k is equal to zero. Since m and k are integers, then if follows that 

, rim 

the coefficient of x K in the power series expansion of bx + ln(l — bx) vanishes for m > [jrJ + 1- 
This gives the following equality: 

LSI 

L k = [x k }L(x) = -j— E am (Pi b) • [ic fc ] | bx + ln(l - bx) j 

L m=1 

where the infinite sum we had before turned to be a finite sum in the last equality. Therefore, the 
power series expansion of bx + ln(l — bx) yields that 


Lk — 


1 


1 — p 


L~J ( 

e|(—( p. »)■{[**] ( 

m= 1l L V 


( n LSJ 

a(P, b) 

1 — p 


b 2 x 2 b 3 x 3 6 4 x 4 

~~2 1 3 1 4~ 

b 2 x 2 b 3 x 3 6 4 x 4 


E (- 1 )” 1 - 1 V 6) • {[- fc ] (*f + ■ ■ ■) } 

m =1L ^ ' ' ' 


From the last equality, we obtain that 


Lk — 


a{ [ M bl E 

L m=l 


A; = 2,3,4,... 


(B.3) 
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where 


m= 1,2,, L^J 


(B-4) 


Cm,k 


A 


E 

q + 22 T • • ■ T im = k 

*1 j ^2 j ■ ■ ■ j im E 2 


*1 *2 ■ • • 


and in the RHS of (IB. 41) designates the number of different permutations of the sequence 

{*i,Z 2 , ■ ■ ■ ,*m}- From the duality between R and L in our example (see (l45l) and (|4fiJ) i. then for 
the calculation of the coefficient R *. in the power series expansion of R we only need to replace p 
in (IB. 31) by 1 — p, so 


m= 1 


p 


i,k ' & 


„ 771—1 


(1 -P,6)} , 


A: = 2,3,4,... 


(B.5) 


Since A(x) = , then Afc = XTyfy- From (IB.21) and (l49l) . we obtain the equality 4/(1) = 

(i-p)(i-b ) 1 an d then it follows from (IB. 31) and the last two equalities that 

L-J 

\ k = {1 ~ b) z bk 2 X|{(-l) m - 1 c m , fc -a m - 1 (p,6)} , = 2,3,4,... . (B.6) 

P m=l 

From the duality between © and ©, then by switching p in JHU) by 1 — p, we obtain that 

L-J 

Pk = {l ~ ( b) _ kb T 2 E {(- 1 ) m_1 c m,k • a m -\l -p,b)}, = 2,3,4,... (B.7) 

t P) m =1 

so (IB.61) and (IB. 71) provide explicit expressions for the coefficients of the d.d. pair from the edge 
perspective. 


B.l A Recursion for the Sequence {c m) k} 


We intend to give now an efficient way to calculate the coefficients c m ^ where 1 < m < |_|j (as 
otherwise c m ./- is equal to zero). To this end, we can rewrite c m ^k in (IB.41) as 


C-m,k 


A 


E 

*1 + *2 + ■ ■ ■ + im = k 
ii 42 j ■ ■ • j im E 2 


1 

i l i'2 ■ ■ ■ im 


m=l, 2,..., L-J 


(B.8) 


where the difference from (|B.4D is that now the order of the terms in the sequence ■ ■ ■ , i m } 

is relevant, so effectively every term of the form .. . 1 . is counted TV/ times in the sum 

above. We will show that this sequence satisfies a simple recursive equation. First, it is trivial that 

ci,k = yr ■ k = 2,3,... (B.9) 
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Since we impose the constraint i\ +12 + • • • i m — k on the calculation of c m then we can rewrite 
c m ,k in (IB.81) as 


Cm,k 


1 

k 


1 

k 


E 

ii ~\~ i 2 + • • • + im — k 

^1 ? ^2 ? • • • 5 ^ 2 

E 

*1 + *2 + ■ • • + = k 

ill *2) • • • , im — 2 


*1 + *2 + • • • + h 
*1^2 • • • 


1 1 
t—: -:-h t—: -:-h ■ 

?2 *3 • • • 1m 1l 13 • • • 1m 


■ ■ + — 


1\ 12 Id, • • • 1m-1 


Symmetry considerations imply that the sum w.r.t. every term among the m terms above are all 
equal to each other, so the expression of c m ,k can be simplified to 


Cm,fc 


m 

~k 


E 

h T *2 T ■ ■ ■ T im = k 
i\ , *2) *3 • ■ ■ 5 im E 2 


1 

*2 1.3 • • • im 


(B.10) 


Comparing to (IB.81) . the last summation is over all the possible vectors {* 2 , 13 ,..., i m } (i.e., i\ does 
not appear in the term . o A . , but only * 2 , I 3 , • • • ,i m appear there). Since ij > 2 for 1 < j < m, 
then the sum ^ 2+23 + - ■ • i m gets nil possible values between 2 (m — 1 ) and fc —2 (where the inequality 
2 (to — 1 ) < k — 2 is automatically satisfied because of the assumption that k > 2 to, as otherwise 
c m ,k = 0). Since the summations in (IB.81) and (IB.101) are taken w.r.t. all possible combinations 
of {*i, I 2 , • • •, im} when the two constraints in (IB.101) are satisfied, then we obtain the following 
recursive equation: 

k—2 

Cm,k — Cfn-1 j • (B.ll) 

j=2(m—1) 

The combination of the recursive equation in (lB~TTl with the initial values in on gives an efficient 
way to calculate the terms of the sequence {c m ^}- We implemented this algorithm in software. 


B.2 An Algorithm for Calculating the Degree Distributions in Section 16.11 

We provide here an algorithm to calculate the coefficients in the d.d. pairs ( L,R ) and (A ,p). This 
algorithm was implemented in software. 

1. Calculate the sequence {c mi fc} with the recursive equation (IB.Ill) and the initial values in (El. 

2. Calculate a(p,b ) from (IB. 21) . 

3. Calculate { L *.} and {A*,} from El and (IB.61) . respectively. 

4. Calculate a(l — p,b) from (IB. 21) . and and {pk\ from El and El, respectively. 


B.3 Asymptotic Expressions for the Degree Distributions in Section 16.11 

We wish to find here asymptotic expressions for the sequences { L *,}, {A k} and { pk } where k 

is sufficiently large. First we see that it is enough to solve the problem for the sequence {L*.}, since 
if there exists a sequence of functions {gk(-, •)} so that Lk ~ gkip, b) for k 3> 1, then we obtain from 
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(m-flR7i) that the following asymptotic expressions are valid for the other degree distributions 
when k 1: 


(1 — b)k (1 — b)k 

Rk ~ 9ki} ~P,b), b2p2 • 9k(p,b ), p k k, b2 2 ' gfcl 1 ~Pi h ) • ( B - 12 ) 

We shall therefore focus on the derivation of the asymptotic behavior of the sequence { L *.} in the 
power series expansion of the function L(-) in (1451) . To this end, we rewrite L(-) in the form 


L(x ) 


bx + ln(l — bx) 

p [6 + ln(l — b )] + (1 — p) [bx + ln(l — 6x)] 

1 /_ p[b + ln(l - b)] _ 

1-p \ p [b + ln(l — b)] + (1 — p) [bx + ln(l — bx)] 

1 ~p { 1 1 + p[ft+ln(l-&)] ' [bx + Hl-bx)]) 

If 1 \ 

1 — p \ 1 — a(p, b) ■ [bx + ln(l — bx)]) 


where the function a is introduced in (113 .2(1 , and is positive for 0 < p < 1 and 0 < b < 1. Let us 
define the function 


h(z\ c) = 


then 


1 — c[z + ln(l — z)] ’ 


l(x) = • ( 1 ~ K hx ; a (p ’ b ))) 


0 < c < 


13 - v/6l 
9 


0.5766. 


(B.13) 


(B.14) 


1 — p 

where the restriction on c follows from the restriction on a that we obtained earlier so that Lq is 
non-negative, and from the equality c = a(p , b) which holds by comparing (EH) and El- 


The following equality therefore holds for k > 2: 

Lk = - - h k (c), c = a(p,b) (B.15) 

1 ~ P 

where {hk(c)}k >o is the sequence the coefficients in the power series expansion h(z ; c) = Y1T= o b k(c)z k 
In the continuation, we will find the asymptotic behavior of the power series expansion of the func¬ 
tion h(z\c) in (IB. 131) . and then use (IB. 151) to derive the asymptotic behavior of the sequence {Lk}, 
and use El for the derivation of the asymptotic behavior of the other degree distributions. 

In |2n[, a class of methods is presented which enables one to translate, on a term-by-term basis, 
an asymptotic expression of a function around a dominant singularity into a corresponding asymp¬ 
totic expansion for the Taylor coefficients of the function. In the continuation of the asymptotic 
analysis, we rely on m- The function h(z; c) has singularities at the points where the following 
equation is satisfied: 

1 — c{z + ln(l — z)) = 0. 

The closed form solution of the above equation is z = z\p where 

z\ = 1 + W(-e = -1 ) , z 2 = z{ 


and W denotes the Lambert W-function. Since we require that 0 < c < 13 9 v/ ^ ~ 0.5766, then z\ 
and z 2 have an absolute value which is at least equal to 2.074 (we note that the absolute value of 
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z\‘i is a monotonic decreasing function of c, so it is achieved when c gets its maximal value within 
this interval). The dominant singularity of the function h(z ; c) is therefore at the point z = 1 where 
the logarithm in (IB. 131) becomes singular. In the region around the dominant singularity at z = 1, 
the function h(z~,c ) behaves like 


h(z; c) = O In 


-i 


1 - z 


1 


so from m, the asymptotic behavior of the coefficients in the power series expansion of this function 
is 

h k = O 


(B.16) 


k hr (&) 

A more careful analysis which is based on the singularity analysis in m shows that 


h k — 


1 -c Jfc(l + dln(A:))‘ 


-1 + 


2 7 


+ 


7 r 2 — 6y 2 


+ o 


1 + dln(k) 2(l + dln(fc)) J \kln 5 (k) 


where d = and 7 ~ 0.5772 designates Euler’s constant. From mm and mm . we obtain 
that 


L k = 


_1_ b^_ _1_ /_ 2 7 \ +Q f b k 

(l - a(p, b)) (1 - p) ‘ k ' (l + d(p> b ) In(*)) 2 ' V 1 + d{p , b ) ln(fc )) + U ln 4 (fe) 


b k 


(B.17) 


where the transition from mm to mm follows by substituting c = a(p,b ) where «(•,•) is 
introduced in (lR2l) . and then based on (fir~ 2 i) and mm 

a(p, b ) 1 - P 


dip, b) = 


1 — a(p,b) 1 — p + p\b + \n.{l — b)\ 


The asymptotic expression for the other degree distributions (i.e., {-Rfc}, {A^} and {p k } fol¬ 
low now immediately from the asymptotic behavior of {L k } in (IB. 171) and the transition to the 
asymptotic behavior of the other d.d. in (lBT2l) . The following degree distributions are therefore 
obtained: 

_ _ 1 _ b^ _1_ /_ 2 7 \ / b k \ 

(l-a(l-p,b))p' k ' (l +d (l-p,b)Hk)) 2 ^ l + d(l-p,b)ln(k)J + {kln\k)J 

\ — 1 — b b k ~ 2 ( 27 \ ^ ^ / b k \ 

(l -a(p,b)^)p 2 (l-p) (l + d(p,b)ln(k)y ^ 1 + dip, b) ln(k)) \ln 4 (k)J 

(l - «(1 ~P, 6))(1 ~P) 2 P ^1 + d(l -p,b)ln(k)y ^ 1 + d(l - p, b) \n(k)) Vln 4 (fe) 

Therefore, the asymptotic behavior of the d.d. pairs w.r.t. the nodes and the edges is given by 

Lk ' Rk = 0 (fctfOfc)) ’ Xk ' pk = 0 (h 7{k)) • 

The parameter b above is chosen according to Gl, so it is determined as a function of the 
erasure probability (p) of the BEC (0.9304 < b < 1). The closer is the value of p to one-half, then 
the smaller becomes the value of 6 , and this accelerates the convergence rate to zero of the d.d. 
pairs above. It is therefore clear from these two equations that the convergence rate of the d.d. 
pairs is improved when p becomes closer to one-half (as is also shown in Fig. 0. 
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Appendix C 

A Generalization of Polya’s Criterion to Discrete Distributions 
and Its Application to Non-Negativity Proofs 

C.l Polya’s Criterion 

Polya’s Criterion nn p. 509] is a simple condition which is sufficient to imply that a function F{t) 
is the inverse Fourier transform (i.e., a characteristic function) of a non-negative function f(x). It 
requires that the function F is real, symmetric, non-negative, and convex non-increasing on [0, oo). 

This gives a simple method of proving that some functions are indeed characteristic functions. 

Example C.l. Let be a sequence of random variables with characteristic function 

F(i) = e” |t|Q , a 6 ( 0 , 1 ]. 

Since there is no closed-form expression for the Fourier transform 

m = 2-J 

we cannot verify directly that f(x) > 0. It is easy, however, to verify that F(t) satisfies Polya’s 
Criterion for a 6 (0,1] and this implies that each Xj is a well-defined random variable. 

C.2 A Generalization of Polya’s Criterion to Discrete Distributions 

In this section, we generalize Polya’s Criterion to discrete distributions and offer an elementary proof 
which is substantially different from the standard “tent function” proof outlined in |211 p. 505]. 

Definition C.l. A function 7 ( 2 ;) has cosine symmetry if 

7(2;) = 7 ( 27 t + x), ViGM 

and 

7 ( 2 ;) = — 7 (vr — x) = —■ 7 ( 7 r + x) = 7 ( 27 r — x) > 0, V x 6 [0, 7 r/ 2 ]. 

Lemma C.l. Let 7 ( 2 :) be a function with cosine symmetry which is non-negative on [0, ^], and 
/(x) be a real function convex for x 6 [0,27r). Then, the integral 'y(x)f(x)dx is non-negative. 

Proof. First, rewrite the integral as 

/*27T /»7t/2 

/ 7 (x)f(x)dx = / ['yi.x)f(x) + 7 ( 7 r — x)f(ir — x) + + x)f(ir + x) + 7(277 — x)f(2n — x)\ dx 

Jo Jo 

/2 

l{x) [f{x) - /(tt -x)~ /(tt + x) + /(2 tt - X )] dx 

/2 

7 (x)g(x)dx 
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where g(x) = f(x) — f{ir — x) — /(tt + x) + /( 2ir — x). Taking the derivative of the function g gives 

g'{x ) = f'{x) + /'(7T - X) ~ f'(-K + X) ~ /'(27T - X) 

= [f(x) - /'(7T + x)] + [/'(tt - x) - /'(27T - x)] , 

which is non-positive for x £ [ 0 , because f"{x) > 0 for x £ [ 0 , 2 tt] (i.e., this gives for x £ [ 0 , 

that /'(x) < f'(x + tt) and f(r r — x) < / / (27r — x)). Since 5 (^) = 0, this implies that g(x) > 0 
for x £ [0, . Finally, we note that the integral is non-negative because both y(x) and g(x) are 

non-negative for x £ [0, |]. □ 

Lemma C.2. Let y(x) be a function with cosine symmetry which is non-negative on [0, f], and 
let F(t ) be a real symmetric function convex for t £ [0, a]. Then, for x > 0 

1 r a 1 r a 

— / F(t)^(tx)dt > — / F(t)'y(tx)dt. 

TT J 0 TT J 22 L I I 

X L 2-7T J 


Proof. Using Lemma lC.1l we find that the integral over each full cycle of y(x) is non-negative which 
leaves only the remaining partial cycle near a. We show this using, for x > 0, the decomposition 


IfM-ir 2 ^ 0 '+i) 

\ r a \ r a \ l 2 ?r j /»——l 

- / F(t)'y(tx)dt = — F(t)-y(tx)dt -|— / F(t)^(tx)dt 

K Jo VTJ 27 r|^i| TT ^ J 2 jtI 


3=0 
I — 1-1 

L 2 ttJ 1 r r2n 


1 r a 1 L _ j r p ztt 

— / F(t)^(tx)dt -\ -/ F 

TT J an I I TTX ^ J 0 

x L 27 t J 


i=0 


27TJ + It 


7 (u)du 


1 


/ F(t)'y(tx)dt. 

7T I 2 -it \ xa \ 

J X L 27T J 

The last step follows from Lemma 1C.II using the fact that F i s convex for u £ [0, 27t] if 

0 < j < LfJ - 1. □ 


Definition C.2. A function F(t) : R —* M satisfies Polya’s Criterion for t £ [0, a], if and only if it 
satisfies the following conditions: 


1. F(t) = F(—t), i.e., F is a symmetric function. 

2. F(t) > 0 i.e., F is non-negative over the interval [0, a]. 

3. F'(t ) < 0, i.e., F is non-increasing over the interval [0,a]. 

4. F"(t) > 0, i.e., F is convex over the interval [0, a]. 

Let V a be the set of functions satisfying Polya’s Criterion on [0, a]. 

Theorem C.2. Any function F £ Voo is the characteristic function of a continuous probabil¬ 
ity distribution, and any function F £ TV which is 27r-periodic (i.e., F{x) = F(x + 2n)) is the 
characteristic function of a discrete probability distribution. 


Proof. For the F £ Voo case, we have 

1 

27T 

1 

r 00 

/(*) 

/ F(t)e~ itx dt 

J — OO 
r 00 


j F(t) cos(tx)dt. 


TT J 

0 
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First, we note that /(0) > 0 because F(t) > 0. Next, we apply Lemma 1C.21 to this with j(x) = 
cos(x), which gives 


f(x) > lim 


CL—> OO 7T / 27T I xa 


I xa I 

1-271- J 


F(t) cos (tx)dt 


> 


lim [ a — 


7T a —KX) 


> — lim — F 

a—>oo x 


2 vr 


2 vr 


xa , . 

, mm F (t)cos(tx) 

2vrJy <t< a 


xa 


x \-2 tt 


where the last step follows since the function F E Voo is non-negative and monotonic non-increasing, 
so for any a > 0 and x > 0 


/ 2 tt 

xa 

] , V t E 

[ 2 t r 

xa 


— 


— 


, a 

V * 

L27tJ 

J 

x 

l2it- 



If lim^oo F(a) = 0, then we have f(x) > 0 for x ^ 0. On the other hand, if lim^oo F(a) > 0, 
then f(x) has a Delta function at x = 0. Subtracting this Delta function returns us to the first 
case and shows that f(x) > 0 for x ^ 0 . 


For the 27r-periodic F E V n case, we have 

h k = i- J* F(t)e~ ikt dt 

i r 

= — F(t) cos(kt)dt. 

ft Jo 

For k = 0, we see that ho > 0 by the non-negativity of F(t) for t E [0, 7 r]. For k > 0, we apply 
Lemma with y(x) = cos(x) to get 


h k — 



F(t ) cos (kt)dt. 


When k is even, we have h k > 0 because the range of integration becomes zero. When k is odd, we 
have 


7r(fc —1/2) 

1 f — n — 1 

h k > — / F(t) cos(tk)dt + 


7T / n(k-l) 
k 

7T - 

1 p 

VT Jn 


7T / 7r(fc-l/2) 

k 


Ti (k — 1) 


+ M — F (it — u) 


F(t) cos {kt)dt 


cos (ku)du. 


This gives h k > 0 because cos{tk ) > 0 for t E [0, and F(t) is non-increasing for t E [0, 7 r ]• D 

Corollary C.l. Let the function 5 : C —> C be analytic on the unit disc except possibly x = 1. If 
the real function h(x) = Re {5 (e“)} is symmetric, convex on [0, 7 r], and satisfies Jj/ h(x)dx > 0, 
then the function g has a power series expansion about zero with non-negative coefficients. 


Proof. By the analyticity of g, we have the power series expansion g(x) = Yl^=o 9nX n ■ Straightfor¬ 
ward algebra shows that 


OO 

h(x) = ^^{Re(g n ) cos(nx) — Im(^ n ) sin(nx)}. 

n =0 


40 






















Since h is assumed to be symmetric, then the imaginary part of g n must be zero for all non-negative 
integers n. Hence, g(x) is real for x G [—1,1). Adding the additional assumption that h is convex 
on the interval [ 0 , 7 r], implies that h is monotonic non-increasing in this interval. To show this, we 
calculate 

h!(x) = Re {ie lx g'(e lx )} => h! ( 7 r) = Re {— ig'(— 1)} = 0. 

Hence, since h"(x) > 0 for x G [0, vr] and h! (it) = 0, then h!(x) < 0 for x G [0, 7 r] ; this yields 
that h is a monotonic non-increasing function over [0, 7 r]. Now, we can apply Theorem 1C.21 to the 
27r-periodic function h. 

Although h(x ) is convex, non-increasing for x G [0, 7 r] and symmetric, we note that it may not 
satisfy the condition h(x ) > 0 for x G [0,7r]. Revisiting the proof of Theorem 1C.21 shows that the 
condition h(x) > 0 over the interval [ 0 , 7 r] can be replaced by the weaker condition f^ h(x)dx > 0 
which holds by assumption. This shows that g n > 0 for all integers n > 0. □ 


C.3 Applications: Non-Negativity Proofs for the Degree Distributions of Some 
Capacity-Achieving Ensembles over the BEC 

In the held of codes on graphs, the validity of the construction of ensembles involves the verification 
that their degree distributions have non-negative positive power series expansions about x = 0 . 


C.3.1 Non-Negativity Proof for the Ensemble of Check-Regular NSIRA Codes 

In Theorem 2], a capacity-achieving ensemble of non-systematic irregular repeat-accumulate 
(NSIRA) codes was introduced, achieving the capacity of a BEC under iterative message-passing 
decoding with bounded complexity per information bit. The d.d. of interest is given by Eq. P5|). 
and the construction is only valid for a particular erasure rate, p , if g(x) = A(x) has a non-negative 
power series expansion for that p. Using Corollary EU we can define h(x) = Re{p(e* x )} and 
verify numerically, for any p, that h"(x) > 0 for x G [0, 7 r], and also h(x) dx > 0. This approach 
appears to work for all p G [0,1), hence it verifies the non-negativity of the power series expansion of 
the left d.d. A. In [3J Appendix C], the non-negativity of the power series expansion of A was proved 
for p G [0, 0.95] where the proof there was quite involved. This approach extends the verification of 
the non-negativity of the power series expansion of A for all 0 < p < 1, thus proving Conjecture 2 
in P Section 3]. 

C.3.2 Non-Negativity Proof for the Ensemble of Bit-Regular ARA Codes 

Starting from (1351) . let 

A R'(x) 

g ( x ) = - . 

x 

Since from (EHT1) . p _ 1 (0) = 0, then p(0) = 0, so the first non-negative coefficient of the power series 
expansion of p(x) about x = 0 is the one of x. Since Q(x) in (l35l) is calculated by an appropriate 
scaling of the integral of p over the interval [0, a;] so that Q( 1) = 1, then the first non-negative 
coefficient of the power series expansion of Q(x ) is the one of x 2 . Finally, from (1311) . it follows that 
also the first non-negative coefficient of the power series expansion of R(x ) about x = (1 is the one 
of x 2 . Hence, since R( 0) = R!( 0) = 0, then R(x) has a power series expansion about x = 0 whose 
all coefficients are non-negative if and only if the function g(x) possesses this property. 
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Using Corollarv lC.il let us define h(x) = Re{^(e* a: )}. It was verified numerically that if p < 0.26, 
then h"{x ) > 0 for x 6 [0,7r], and also j Q h(x) dx > 0. This therefore proves that if p < 0.26, then 
the function g{x) (and hence, also the d.d. R{x)) has a power series expansion about x = 0 whose 
all coefficients are non-negative. 


C.3.3 Non-Negativity Proof for the Ensemble of Self-Matched ARA Codes 


In Section EH we construct capacity-achieving ARA ensembles from self-matched LDPC codes. 
From (ESI and m, the left and right degree distributions can be rewritten in the form 


L(x) 
R{x) 


it i (— bx — ln(l — bx)) 

1 + ci (—bx — ln(l — bx)) ’ 
K 2 (—bx — ln(l — bx)) 

1 + C 2 (—bx — ln(l — bx)) 


(C.l) 


where 


K\ = - 

ci = 


1 


p[b + ln(l - b)j' 

1 — p 

p[b + ln(l - b)} ’ 


K-2 = ~ 


C2 = 


1 


(1 -p)[b + ln(l - b )] ’ 
P 


(1 —p)[b + ln(l - b )] 


(C.2) 


Note that K\ 2 and c\p are positive for values of p and b in the range 0 < p < 1 and 0 < b < 1. 
Hence, in order to find the region of p and b where both degree distributions (i.e., L(x) and R(x)) 
have non-negative power series expansion about x = 0, let us find conditions on the parameter c 
which ensure that 


g(x) 


—x — ln(l — x) 

1 + c (—x — ln(l — x)) 


(C.3) 


has a non-negative power series expansion about x = 0. The first few terms of the expansion are 


g(x) 


7 


^2g n (c)x n 

71=2 


3x 2 + 2x 3 + (1 - c)x 4 + (3 - 5c)x 5 + (12 - 26c + 9c 2 )x 6 + (120 - 308c + 210)x 7 + Q ^ 


6 


15 


70 


840 


Applying Corollary ED directly to this function does not work because the function is neither 
non-increasing nor convex. Instead, we remove the first few terms of the power series and apply 
Corollary 1C. II to 


g(x) 


1 ( —x — log(l — x) 

X 8 l 1 + c [ — x — log(l — x)} 



Following the notation in Corollary EH let the function h be 


h(x) A Re{„( e “)} 

= i[9(e“) + 9(e-“) 


Numerically, we find that if c E [0, 0.6], then h"{x) > 0 for x E [0,7r], Checking the first few terms 
by hand shows that ge(c) becomes negative first for c > c* = 13 ~ 9 v/ ^ ~ 0.5766. Therefore, we find 
that the original power series expansion of the function g in dOll is non-negative if and only if 
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c G [0, c*]. From (Kill) . it follows that the left and right d.d. (L(x) and R{x), respectively) have 
non-negative power series expansion about x = 0 if and only if 

13 - v/6l 13 - v/6l 

ci <--—, c 2 <--— 

9 ’ 9 

which based on (EH, is translated to the condition in m- 

Finally, we consider the question of whether the numerical verification of convexity on [0,7r] is 
reasonable. While one could also compute any finite number of terms in the power series expansion 
and verify their non-negativity, we note that the next term could always be negative. This new 
approach is different because, in many cases, the convexity can be verified numerically in finite 
time using complex interval arithmetic. The basic approach is subdividing the interval [0, vr] into a 
large number of small overlapping intervals. If interval output of the function h evaluated on each 
subinterval of [0,7r] is non-negative, then this proves that h is convex on [0,7r]. If h also depends 
on some parameter, then the parameter interval can also be subdivided into small overlapping 
subintervals. If the first test succeeds for each parameter subinterval, then we have shown that h 
is convex on [0, ir\ for all parameter values as well. 
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ARA vs. IRA vs. LDPC n=8192 Rate 0.5 



ARA vs. IRA vs. LDPC n=65536 Rate 0.5 



Figure 8: Simulations for the ensembles of ARA and NSIRA codes constructed from LDPC codes 
with self-dual degree distributions (see Sections Ki.ll and 16.81) , and the ensemble of right-regular 
LDPC codes in |2j. The plots refer to block lengths of 8192 and 65536 bits (see upper and lower 
plots, respectively) and a design rate of 0.5 bits per channel use. No high-rate outer code is assumed. 
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ARA vs. IRA vs. LDPC n=8192 Rate 0.5 



ARA vs. IRA vs. LDPC n=65536 Rate 0.5 



Figure 9: Simulations for the ensembles of ARA and NSIRA codes constructed from LDPC codes 
(see Sections Kill and EH, and right-regular LDPC codes [2]. The plots refer to block lengths of 
8192 and 65536 bits (see upper and lower plots, respectively) and a design rate of 0.5 bits per 
channel use. Since the ensemble averaged performance is simulated, high-rate outer codes (rates 
|j|| and HH§, respectively) are used to lower the error floor due to small stopping sets. These 
outer codes are chosen uniformly at random from the ensemble of the binary linear block codes and 
their rate loss is neglected. 
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ARA Self-Matched Rate 0.5 



Erasure Probability (p) 


ARA Self-Matched Rate 0.5 



Erasure Probability (p) 


Figure 10: Simulations for the ensemble of ARA codes in Section 16.11 constructed from the 
ensemble of LDPC codes whose degree distributions are matched to themselves. The design rate of 
the ensemble is 0.5 bits per channel use. The upper plot refers to the case where there is no outer 
high-rate code, and the lower plot refers to the case where there is such a code. 
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ARA Self-Matched Rate 0.7 



Erasure Probability (p) 


ARA Self-Matched Rate 0.5 Punctured to Rate 0.7 



Erasure Probability (p) 


Figure 11: Simulations for the ensemble of self-matched ARA codes whose rate is 0.7 bits per 
channel use, having high-rate outer codes (the rate of the outer code is ^|, |j|| and |||||, ^ or a 
block length of 1024, 8192 and 65536 bits, respectively.) The upper plot refers to the case where 
the ensemble of self-matched ARA codes is directly designed for a rate of 0.7 (without puncturing), 
and the lower plot refers to the design of the self-matched ARA ensemble for a rate of 0.5, and 
then increasing the rate to 0.6 by random puncturing of the code bits. 


48 


























Punctured vs. Non-Punctured ARA Self-Matched n=8192 Rate 0.7 



Punctured vs. Non-Punctured ARA Self-Matched n=65536 Rate 0.7 



Erasure Probability (p) 


Figure 12: Simulations for the ensemble of punctured ARA codes in Section ffi. 1 1 where we compare 
the case where the self-matched ensemble is designed directly to a rate of 0.7 bits per channel use, 
versus the case of designing the ensemble for a rate of 0.5 and increasing the rate by puncturing. 
The upper and lower plots refer to block lengths of 8192 and 65536 bits, respectively. P and NP 
stand for ’punctured’ and ’non-punctured’. 
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Figure 13: Simulations for the ALDPC-BR3 ensemble (i.e., bit-regular accumulate-LDPC ensemble 
whose bit nodes have degree-3); the design rate of this ensemble is set to j bits per channel use. 
The upper plot refers to the case where there is no high-rate outer code, and the lower plot refers 
to the case where there is such an outer code. The plots refer to block lengths of 1024, 8192 and 
65536 bits; for the lower plot and these block lengths, the rate of the outer random code is equal 

respectively. 
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